BYOK Multi-Tenant AI: Bring Your Own Key Architecture

TL;DR — BYOK (Bring Your Own Key) lets multi-tenant AI platform customers supply their own LLM provider API keys. osFoundry identifies three patterns: Gateway (server holds key, sees all traffic), Embedded SDK (client holds key, server never sees payload), Hybrid (key stored server-side, call from client). DVARA implements a 4-step credential resolution chain: tenant credential → platform-default → vault → env var. Keys encrypted with AES-256-GCM with per-tenant key derivation — "one tenant cannot decrypt another tenant's BYOK ciphertext." Meridian Blue supports allowedModels restrictions and per-request BYOK override logging. Gartner: 64% of enterprise GenAI buyers already hold direct provider contracts. Benefits: enterprise discounts, data sovereignty, key isolation, tenant-controlled lifecycle. Integrate with multi-tenant architecture, white-label platform, cost tracking, and zero-trust.

osFoundry defines BYOK: "a product architecture where customers supply their own API keys for AI providers — OpenAI, Anthropic, Google, or self-hosted models — and the product routes calls through those keys rather than a centralized platform account."

They note: "A 2025 Gartner snapshot put 64% of enterprise GenAI buyers as already holding direct provider contracts. Re-billing them through your account is friction."

BYOK Architecture

BYOK Architecture Patterns

Credential Resolution Chain

Implementation

from cryptography.hazmat.primitives.ciphers.aead import AESGCM
import os
import hashlib
import base64

class BYOKCredentialManager:
    """Multi-tenant BYOK credential management with AES-256-GCM."""

    def __init__(self, db, master_key: bytes, vault_client=None):
        self.db = db
        self.master_key = master_key  # HSM or KMS-derived
        self.vault = vault_client  # Optional: HashiCorp Vault, AWS SM
        self.cache = {}  # In-memory cache (TTL: 5 min)

    async def register_key(self, tenant_id: str, provider: str,
                           api_key: str, allowed_models: list[str] = None):
        """Tenant registers their own API key."""
        # Derive per-tenant encryption key
        tenant_key = self._derive_tenant_key(tenant_id)

        # Encrypt with AES-256-GCM
        aesgcm = AESGCM(tenant_key)
        nonce = os.urandom(12)
        ciphertext = aesgcm.encrypt(nonce, api_key.encode(), None)

        # Store encrypted key (plaintext never persisted)
        stored = f"{nonce.hex()}:{ciphertext.hex()}"

        await self.db.insert("tenant_credentials", {
            "tenant_id": tenant_id,
            "provider": provider,
            "encrypted_key": stored,
            "allowed_models": allowed_models,
            "active": True,
            "created_at": datetime.utcnow(),
        })

        # Invalidate cache
        self.cache.pop((tenant_id, provider), None)

        # Audit log
        await self.audit.log(
            tenant_id=tenant_id,
            event="byok_key_registered",
            provider=provider,
        )

        return {"status": "registered", "provider": provider}

    async def resolve_credential(self, tenant_id: str, provider: str,
                                  model: str) -> str:
        """Resolve API key through credential chain."""

        # Check cache first
        cache_key = (tenant_id, provider)
        if cache_key in self.cache:
            cred = self.cache[cache_key]
            if self._model_allowed(cred, model):
                return cred["api_key"]

        # Step 1: Tenant credential
        cred = await self.db.find_one("tenant_credentials", {
            "tenant_id": tenant_id,
            "provider": provider,
            "active": True,
        })

        if cred:
            # Check model restrictions
            if cred.get("allowed_models") and model not in cred["allowed_models"]:
                raise ModelNotAllowedError(
                    f"Model {model} not allowed for this credential"
                )

            # Decrypt
            api_key = self._decrypt(tenant_id, cred["encrypted_key"])

            # Cache
            self.cache[cache_key] = {"api_key": api_key, "allowed_models": cred.get("allowed_models")}

            return api_key

        # Step 2: Platform default
        platform_cred = await self.db.find_one("tenant_credentials", {
            "tenant_id": None,  # Platform-level
            "provider": provider,
            "active": True,
        })

        if platform_cred:
            api_key = self._decrypt("platform", platform_cred["encrypted_key"])
            return api_key

        # Step 3: Vault
        if self.vault:
            vault_key = await self.vault.get_secret(
                path=f"ai-platform/{provider}",
                tenant_id=tenant_id,
            )
            if vault_key:
                return vault_key

        # Step 4: Environment variable
        env_key = os.environ.get(f"{provider.upper()}_API_KEY")
        if env_key:
            return env_key

        # No credential found
        raise CredentialNotFoundError(
            f"No credential for tenant {tenant_id}, provider {provider}"
        )

    def _derive_tenant_key(self, tenant_id: str) -> bytes:
        """Derive per-tenant encryption key from master key."""
        # Salt tenant_id into key derivation
        derived = hashlib.pbkdf2_hmac(
            "sha256",
            self.master_key,
            tenant_id.encode(),
            iterations=100000,
            dklen=32,
        )
        return derived

    def _decrypt(self, tenant_id: str, encrypted: str) -> str:
        """Decrypt API key with per-tenant key."""
        nonce_hex, ciphertext_hex = encrypted.split(":")
        tenant_key = self._derive_tenant_key(tenant_id)

        aesgcm = AESGCM(tenant_key)
        plaintext = aesgcm.decrypt(
            bytes.fromhex(nonce_hex),
            bytes.fromhex(ciphertext_hex),
            None,
        )
        return plaintext.decode()

    async def rotate_key(self, tenant_id: str, provider: str, new_key: str):
        """Rotate tenant's API key."""
        # Overwrite with new key
        await self.register_key(tenant_id, provider, new_key)

        # Invalidate cache
        self.cache.pop((tenant_id, provider), None)

        await self.audit.log(
            tenant_id=tenant_id,
            event="byok_key_rotated",
            provider=provider,
        )

    async def revoke_key(self, tenant_id: str, provider: str):
        """Revoke tenant's API key."""
        await self.db.update("tenant_credentials", {
            "tenant_id": tenant_id,
            "provider": provider,
            "active": True,
        }, {"active": False, "revoked_at": datetime.utcnow()})

        self.cache.pop((tenant_id, provider), None)

        await self.audit.log(
            tenant_id=tenant_id,
            event="byok_key_revoked",
            provider=provider,
        )

BYOK vs Platform-Managed Keys

BYOK Multi-Tenant AI Checklist

• [ ] Implement credential resolution chain: tenant → platform-default → vault → env
• [ ] Encrypt tenant API keys with AES-256-GCM
• [ ] Derive per-tenant encryption keys (tenant_id salted into derivation)
• [ ] Verify one tenant cannot decrypt another's ciphertext (integration test)
• [ ] Never log plaintext API keys — decrypt only on request hot path
• [ ] Support multiple providers: OpenAI, Anthropic, Google, self-hosted (vLLM)
• [ ] Implement allowedModels restriction per credential
• [ ] Set up key registration endpoint (admin only per tenant)
• [ ] Implement key rotation (overwrite encrypted blob, no recovery path)
• [ ] Implement key revocation (deactivate, fall through to platform-default)
• [ ] Configure strict BYOK mode (HTTP 403 if no tenant credential)
• [ ] Set per-tenant metadata override for tiered BYOK (free vs paid)
• [ ] Pair strict BYOK with API key enforcement (prevent anonymous bypass)
• [ ] Support vault reference mode (HashiCorp Vault, AWS Secrets Manager, Azure KV)
• [ ] Implement credential caching (TTL 5 min) for performance
• [ ] Invalidate cache on key rotation and revocation
• [ ] Log BYOK usage per request ("BYOK used for this request")
• [ ] Implement per-tenant rate limiting (independent of key)
• [ ] Implement per-tenant token quotas (independent of provider billing)
• [ ] Support multi-provider routing (OpenAI primary, Anthropic fallback)
• [ ] Integrate with multi-tenant architecture
• [ ] Integrate with white-label platform reseller hierarchy
• [ ] Set up per-tenant cost tracking for platform fees
• [ ] Apply zero-trust architecture to key management
• [ ] Apply OWASP LLM Top 10 security controls
• [ ] Consider gateway vs embedded SDK vs hybrid based on compliance needs
• [ ] Set up platform monitoring for credential health
• [ ] Alert on: missing credentials, expired keys, decryption failures
• [ ] Consider self-hosted AI for on-prem keys
• [ ] Document BYOK onboarding for tenants
• [ ] Test key rotation without service interruption
• [ ] Test key revocation and fallback behavior
• [ ] Test cross-tenant isolation (tenant A cannot use tenant B's key)
• [ ] Quarterly security audit: review key access logs, rotate master key
• [ ] Consider HSM or KMS for master key storage
• [ ] Implement graceful error handling for expired or invalid keys

FAQ

What is BYOK in AI platforms?

BYOK (Bring Your Own Key) is a product architecture where customers supply their own API keys for AI providers (OpenAI, Anthropic, Google, self-hosted models) and the platform routes calls through those keys rather than a centralized platform account. The customer is billed by the provider directly; the platform charges a platform fee on top. Benefits: (1) Enterprise buyers use their negotiated rates and committed spend. (2) Regulated industries maintain direct data-processing agreements with providers. (3) Compromised key affects only one tenant. (4) Tenants control their own key lifecycle (rotation, revocation). (5) Platform provider doesn't pay for LLM calls. A 2025 Gartner snapshot put 64% of enterprise GenAI buyers as already holding direct provider contracts.

What are the BYOK architecture patterns?

Three BYOK architecture patterns: (1) Gateway — every model call goes through your servers. You hold the customer's key in escrow, decrypt it per request, sign the upstream call. You see every prompt and response. Best for centralized observability and fallback orchestration. (2) Embedded SDK — the client (browser, desktop, mobile) holds the key and calls the provider directly. Your servers never see the key or the payload. Best for regulated customers who can't route data through a third party. (3) Hybrid — key stored server-side, but the actual model call originates client-side. Server issues short-lived signed tokens or per-call ephemeral credentials. Best when you need centralized key management but want direct client-to-provider calls.

How do you encrypt tenant API keys?

Encrypt tenant API keys with AES-256-GCM symmetric encryption with per-tenant key derivation. The tenant_id is salted into the key derivation, so one tenant cannot decrypt another tenant's ciphertext. Storage modes: (1) ENCRYPTED — keys stored in database as AES-256-GCM ciphertext, decrypted in-process on every cache-miss request. Simplest to operate. (2) REFERENCE — database holds only a pointer (vault path, AWS Secrets Manager ARN, Azure Key Vault secret name). Platform never holds the secret at rest. Zero-trust posture — even an attacker with full database access gets nothing. Required by many regulated enterprises. The plaintext key never appears in logs and is decrypted only on the request hot path.

What is a credential resolution chain?

A credential resolution chain determines which API key to use for each request. The first step that returns a key wins: (1) Tenant credential — stored for this specific tenant, active and scoped. (2) Platform-default credential — fallback for tenants without BYOK. (3) Vault — HashiCorp Vault, AWS Secrets Manager, or Azure Key Vault, if configured. (4) Environment variable — OPENAI_API_KEY on the gateway container. For strict BYOK mode, the chain stops at step 1 — a tenant with no own credential receives HTTP 403 instead of falling through. For tiered SaaS, free tenants can borrow the platform-default while paid tenants must BYOK via per-tenant metadata override.

How does BYOK affect multi-tenant billing?

BYOK changes the billing model: instead of the platform paying for all LLM calls and re-billing tenants, each tenant pays their provider directly. The platform charges only a platform fee (subscription or usage-based for infrastructure, not tokens). Benefits: (1) No markup on token costs — tenants use their negotiated rates. (2) No billing reconciliation between platform and provider. (3) Tenants see charges directly on their provider invoice. (4) Platform margin comes from platform features (RAG, agents, governance), not token markup. For mixed deployments: free-tier tenants use platform-default key (platform pays), paid-tier tenants use BYOK (tenant pays). Track usage per tenant for platform fee calculation and quota enforcement.

Want a self-hosted AI company brain that does all of this out of the box?
Book a demo →