Protecting LLM API Keys: Fernet Encryption in Dronzer AI Gateway

Building the Dronzer AI Gateway was an ambitious project. The goal was an enterprise-grade orchestration engine capable of routing requests across over 45 different LLM providers, offering bulletproof failover and 100% OpenAI API compatibility. This meant users could bring their own API keys for various models, switching them on the fly, or using keys managed by the platform.

As the number of integrated LLM providers grew, and with each user potentially managing multiple API keys, a critical security question emerged: how do we store and manage these highly sensitive API credentials?

The Problem: Securing a Multitude of Sensitive API Keys

Imagine a system where users can configure their API keys for GPT-4, Claude, Gemini, Groq, Mistral, and dozens of other models. These aren't just arbitrary strings; they're access tokens to powerful, often expensive, AI services. If compromised, they could lead to unauthorized usage, significant billing charges, or even malicious access to AI capabilities. Storing such keys directly in a database, even if the database itself is secured, is a non-starter. It creates a single point of failure and violates fundamental security best practices.

My primary concerns were:

1. Data at Rest Protection: API keys stored in the PostgreSQL database must be encrypted. A breach of the database should not immediately expose all user API keys.
2. Data in Transit Protection: While standard TLS covers network traffic, the internal handling of keys (decryption, usage) needed careful thought.
3. Access Control: Users should only be able to access and use their own API keys. There could be no accidental leakage or unauthorized access between user accounts.
4. Performance: Encryption and decryption operations needed to be efficient enough not to introduce significant latency into the LLM request flow, which was already dealing with external API calls.
5. Key Management: The encryption key itself needed to be managed securely, separate from the encrypted data.
6. Auditability: We needed a way to ensure that keys were being handled correctly and that any access attempts were logged.

Simply put, directly storing API keys felt like leaving the vault door open with a 'come on in' sign. I needed a robust mechanism that balanced strong security with the operational realities of a high-performance API gateway.

Root Cause Analysis: Why Simple Solutions Fall Short

My initial thought process quickly ruled out several naive approaches:

• Plaintext Storage: Absolutely not. This is a catastrophic security vulnerability waiting to happen. Any database compromise would immediately expose all keys.
• Hashing: Hashing is one-way. You can't retrieve the original key, which means you can't use it to make API calls. Not suitable for credentials that need to be used.
• Client-Side Encryption: While tempting, offloading encryption/decryption to the client introduces its own set of problems. How do you manage the client-side encryption key? How do you prevent tampering? It also complicates server-side operations like failover or re-routing, where the backend might need to access keys directly.

The core problem was that the backend needed to access the original, unencrypted API key at runtime to make calls to LLM providers. This meant a reversible encryption method was required, coupled with strong access control mechanisms.

I needed a solution that provided strong encryption for data at rest, ensured that keys were only decrypted in memory for the shortest possible time, and tied access to specific authenticated users.

The Solution: Layered Security with Fernet and JWT

My solution combined two powerful tools: Fernet symmetric encryption for the API keys themselves, and JSON Web Tokens (JWT) for robust user authentication and authorization.

Fernet Encryption for API Keys

Fernet is a symmetric encryption scheme provided by the cryptography library in Python. It's designed for simplicity and security, using a single key for both encryption and decryption. Key advantages for my use case:

• Symmetric: Perfect for encrypting data that the application itself needs to decrypt later.
• URL-Safe Tokens: Encrypted data is base64 encoded, making it easy to store in databases or transmit.
• Time-Limited Validity (Optional): Fernet tokens can include timestamps, adding a layer of protection against replay attacks if used correctly (though for API keys, the primary concern is storage, not transmission).
• Strong Algorithms: It uses AES in CBC mode with PKCS7 padding and HMAC for authenticity, ensuring both confidentiality and integrity.

The process would be:

1. Generate a strong Fernet key and store it securely (e.g., in an environment variable).
2. When a user provides an API key, encrypt it using the Fernet key before saving it to the database.
3. When an LLM call needs to be made, retrieve the encrypted key from the database, decrypt it in memory using the same Fernet key, use it, and then immediately discard the decrypted value.

JWT for User Authentication and Authorization

JWTs are standard for authentication in modern web applications. In Dronzer AI Gateway, they served to:

1. Authenticate Users: Verify the identity of the user making a request.
2. Authorize Access: Ensure that the authenticated user is allowed to access the specific API keys they're requesting or using.

The JWT payload would contain the user ID. This ID is crucial. When a request comes in, the FastAPI backend validates the JWT, extracts the user ID, and then uses that ID to query the database for that specific user's encrypted API keys. This prevents any user from accessing another user's credentials.

Secure Key Management for the Fernet Key

The Fernet key is the master key. If it's compromised, all encrypted API keys are vulnerable. Therefore, its protection is paramount. I opted for storing it as an environment variable in the production environment. For Docker deployments, this means passing it securely. For a VPS, it's set in the systemd service file or through a .env file loaded by the application. It is explicitly not committed to the codebase.

Implementation Details

Here's how this looked in practice within the FastAPI application:

First, I defined a configuration variable for the Fernet key, loaded from environment variables:

# app/config.py

import os
from cryptography.fernet import Fernet

class Settings:
    APP_NAME: str = "Dronzer AI Gateway"
    SECRET_KEY: str = os.getenv("SECRET_KEY", "super-secret-jwt-key") # For JWT signing
    ALGORITHM: str = "HS256"
    ACCESS_TOKEN_EXPIRE_MINUTES: int = 30

    # Fernet key for API key encryption
    FERNET_KEY: str = os.getenv("FERNET_KEY", Fernet.generate_key().decode()) # Generate if not set (DEV ONLY!)

    # Database settings, etc.

settings = Settings()

# Initialize Fernet instance
fernet_instance = Fernet(settings.FERNET_KEY.encode())

Notice the Fernet.generate_key().decode() fallback for FERNET_KEY. This is strictly for development environments. In production, os.getenv("FERNET_KEY") must return a pre-generated, securely stored key.

Next, I created helper functions for encryption and decryption:

# app/security/encryption.py

from cryptography.fernet import Fernet, InvalidToken
from app.config import fernet_instance

def encrypt_api_key(api_key: str) -> str:
    """Encrypts an API key using Fernet."""
    return fernet_instance.encrypt(api_key.encode()).decode()

def decrypt_api_key(encrypted_api_key: str) -> str | None:
    """Decrypts an API key using Fernet. Returns None on decryption failure."""
    try:
        return fernet_instance.decrypt(encrypted_api_key.encode()).decode()
    except InvalidToken:
        # Log this error, it indicates a tampering attempt or wrong key
        print("Warning: Invalid Fernet token encountered.")
        return None
    except Exception as e:
        print(f"Error decrypting API key: {e}")
        return None

The database model for storing API keys would look something like this (using SQLAlchemy):

# app/models/api_key.py

from sqlalchemy import Column, Integer, String, ForeignKey
from sqlalchemy.orm import relationship
from app.database import Base

class UserAPIKey(Base):
    __tablename__ = "user_api_keys"

    id = Column(Integer, primary_key=True, index=True)
    user_id = Column(Integer, ForeignKey("users.id"), index=True)
    provider_name = Column(String, index=True) # e.g., "openai", "anthropic"
    encrypted_key = Column(String, nullable=False) # The Fernet-encrypted key

    user = relationship("User", back_populates="api_keys")

    # ... potentially add other fields like key_alias, is_active, etc.

And finally, the FastAPI endpoints and dependencies to integrate this:

# app/api/dependencies.py

from fastapi import Depends, HTTPException, status
from fastapi.security import OAuth2PasswordBearer
from sqlalchemy.orm import Session
from jose import JWTError, jwt

from app.config import settings
from app.database import get_db
from app.models.user import User # Assuming a User model exists
from app.models.api_key import UserAPIKey
from app.security.encryption import decrypt_api_key

# Standard OAuth2PasswordBearer for JWT token extraction
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="token")

async def get_current_user(token: str = Depends(oauth2_scheme), db: Session = Depends(get_db)) -> User:
    credentials_exception = HTTPException(
        status_code=status.HTTP_401_UNAUTHORIZED,
        detail="Could not validate credentials",
        headers={"WWW-Authenticate": "Bearer"},
    )
    try:
        payload = jwt.decode(token, settings.SECRET_KEY, algorithms=[settings.ALGORITHM])
        user_id: int = payload.get("sub")
        if user_id is None:
            raise credentials_exception
    except JWTError:
        raise credentials_exception
    
    user = db.query(User).filter(User.id == user_id).first()
    if user is None:
        raise credentials_exception
    return user

async def get_user_api_key(provider: str, current_user: User = Depends(get_current_user), db: Session = Depends(get_db)) -> str:
    """Dependency to retrieve and decrypt a user's API key for a specific provider."""
    api_key_record = db.query(UserAPIKey).filter(
        UserAPIKey.user_id == current_user.id,
        UserAPIKey.provider_name == provider
    ).first()

    if not api_key_record:
        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=f"API key for {provider} not found.")

    decrypted_key = decrypt_api_key(api_key_record.encrypted_key)
    if decrypted_key is None:
        raise HTTPException(status_code=status.HTTP_500_INTERNAL_SERVER_ERROR, detail="Failed to decrypt API key.")
    
    return decrypted_key

# Example API endpoint using the dependency
# @router.post("/llm-query")
# async def query_llm(
#     prompt: str,
#     provider: str = Query(..., description="LLM provider name"),
#     api_key: str = Depends(get_user_api_key) # This will get the decrypted key
# ):
#     # Use the 'api_key' directly to make the LLM call
#     llm_client = SomeLLMClient(api_key=api_key)
#     response = await llm_client.generate(prompt)
#     return {"response": response}

This setup ensures that:

• API keys are encrypted before they hit the database.
• Only authenticated users can trigger the decryption process for their own keys.
• Decryption happens in memory, and the plain-text key is used only for the immediate LLM API call.
• The master Fernet key is kept separate and secure.

Lessons Learned

This exercise reinforced several critical aspects of secure software development:

1. Defense-in-Depth is Non-Negotiable: Relying on a single security layer is never enough. Combining database encryption with strong authentication and authorization creates a much more resilient system. Even if one layer is breached, others provide protection.
2. Key Management is Paramount: The security of symmetric encryption hinges entirely on the secrecy of the master key. This isn't just a technical problem; it's an operational one. Proper procedures for generating, storing, rotating, and revoking encryption keys are essential. For Dronzer AI Gateway, this meant strict environment variable management and avoiding hardcoding or committing the key.
3. Trade-offs Exist: While Fernet is straightforward and secure for data-at-rest encryption, it's still symmetric. If the Fernet key is compromised, all data encrypted with it can be decrypted. For extremely high-security scenarios or multi-tenant systems where tenants might need to manage their own keys, asymmetric encryption (e.g., individual public/private key pairs per user) or even hardware security modules (HSMs) might be considered, though they introduce significant complexity.
4. Error Handling for Security: The decrypt_api_key function includes a try-except InvalidToken block. This isn't just good practice; it's a security measure. An InvalidToken exception could indicate data corruption or, more alarmingly, a tampering attempt. Logging these events is crucial for detecting potential breaches.
5. Performance Considerations: While Fernet is fast, frequent encryption/decryption operations can add overhead. For Dronzer AI Gateway, where LLM calls are inherently I/O bound (waiting for external APIs), the overhead was negligible. However, in other high-throughput contexts, profiling is always necessary.
6. Simplicity Aids Security: Fernet's API is simple and hard to misuse, which reduces the chances of introducing vulnerabilities through incorrect implementation. Sometimes, the most complex problem has an elegant, simple solution.

Implementing this robust encryption and access control system was a fundamental step in making Dronzer AI Gateway not just functional, but trustworthy. It ensures that while users leverage the power of diverse LLMs, their sensitive credentials remain protected, a non-negotiable requirement for any enterprise-grade platform.