BYOK Cost Savings: How Bring Your Own Key Reduces Enterprise AI Spending

TL;DR — BYOK (Bring Your Own Key) eliminates token markups and preserves enterprise provider discounts. osFoundry: "64% of enterprise GenAI buyers already hold direct provider contracts. BYOK lets the customer absorb token volatility while you charge a flat platform fee." VerticalAPI: "BYOK gateways charge 0% token markup vs 5% aggregator markup. At $5K/month that's $250/month savings." Three architecture patterns: centralized gateway (observability), embedded SDK (compliance), hybrid (balance). Three pricing models: pure passthrough (flat fee, zero token charge), percentage markup (7.5% on tokens), flat per-call. Gateway-LLM: "Virtual API keys sit in front of provider credentials — a leaked key is a five-second revocation, not a five-hour key rotation." KeyGate: "No proxy, zero added latency — uses vendor Admin APIs to create scoped projects per developer." Choose BYOK when spend exceeds $500-1,000/month. Integrate with BYOK multi-tenant, enterprise TCO, LLM cost optimization, and usage tracking.

osFoundry frames the business case: "Letting users bring their own AI provider keys is no longer optional for serious B2B products. A 2025 Gartner snapshot put 64% of enterprise GenAI buyers as already holding direct provider contracts. Re-billing them through your account is friction. If you charge $0.05 per message and the underlying tokens cost $0.04, your margin is fragile. BYOK lets the customer absorb token volatility directly while you charge a flat platform fee."

VerticalAPI quantifies the savings: "BYOK gateways charge 0% token markup vs managed aggregators at ~5% on top of provider list. At $5K/month spend that is $250/month savings, often more than the BYOK platform fee. BYOK preserves your direct provider contracts — volume discounts, prompt caching commitments, enterprise SLAs."

BYOK Architecture Patterns

BYOK vs Managed Aggregator Comparison

Source: VerticalAPI

BYOK Pricing Models

Implementation

import hashlib
import json
from datetime import datetime, timezone
from dataclasses import dataclass
from typing import Optional

@dataclass
class BYOKManager:
    """BYOK key management with virtual keys, budgets, and governance."""

    def __init__(self, db, redis, vault, audit):
        self.db = db
        self.redis = redis
        self.vault = vault  # secrets manager (e.g., HashiCorp Vault)
        self.audit = audit

    async def store_provider_key(self, tenant_id: str, user_id: str,
                                 provider: str, api_key: str) -> str:
        """Store customer's provider API key encrypted in vault."""
        key_id = hashlib.sha256(
            f"{tenant_id}:{user_id}:{provider}:{datetime.utcnow()}"
            .encode()
        ).hexdigest()[:16]

        # Store encrypted in vault (never in database)
        await self.vault.write(
            f"byok/{tenant_id}/{user_id}/{provider}",
            api_key=api_key,
        )

        # Store metadata in database (not the key itself)
        await self.db.execute(
            """INSERT INTO byok_provider_keys
               (key_id, tenant_id, user_id, provider, created_at, active)
               VALUES ($1, $2, $3, $4, $5, true)""",
            key_id, tenant_id, user_id, provider,
            datetime.now(timezone.utc),
        )

        await self.audit.log(
            tenant_id=tenant_id, user_id=user_id,
            action="byok_key_stored",
            entity_type="provider_key",
            after={"provider": provider, "key_id": key_id},
        )

        return key_id

    async def create_virtual_key(self, tenant_id: str, team_id: str,
                                 models: list, rpm: int = 100,
                                 tpm: int = 100000,
                                 monthly_budget_usd: float = 500.0,
                                 router: str = "classifier") -> dict:
        """Create a virtual API key with governance controls."""
        import secrets
        virtual_key = f"sk-byok-{secrets.token_urlsafe(32)}"
        key_hash = hashlib.sha256(virtual_key.encode()).hexdigest()

        record = {
            "key_id": key_hash[:16],
            "tenant_id": tenant_id,
            "team_id": team_id,
            "key_hash": key_hash,
            "models": models,  # allowlist
            "rpm": rpm,
            "tpm": tpm,
            "monthly_budget_usd": monthly_budget_usd,
            "router": router,
            "active": True,
            "created_at": datetime.now(timezone.utc),
        }

        await self.db.execute(
            """INSERT INTO byok_virtual_keys
               (key_id, tenant_id, team_id, key_hash, models,
                rpm, tpm, monthly_budget_usd, router, active, created_at)
               VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, true, $10)""",
            record["key_id"], tenant_id, team_id, key_hash,
            json.dumps(models), rpm, tpm,
            monthly_budget_usd, router,
            datetime.now(timezone.utc),
        )

        await self.audit.log(
            tenant_id=tenant_id,
            action="byok_virtual_key_created",
            entity_type="virtual_key",
            after={
                "team_id": team_id, "models": models,
                "rpm": rpm, "budget": monthly_budget_usd,
            },
        )

        return {
            "virtual_key": virtual_key,  # shown once
            "key_id": record["key_id"],
            "models": models,
            "rpm": rpm,
            "tpm": tpm,
            "monthly_budget_usd": monthly_budget_usd,
        }

    async def authenticate_call(self, virtual_key: str,
                                model: str) -> dict:
        """Authenticate a virtual key and check governance."""
        key_hash = hashlib.sha256(virtual_key.encode()).hexdigest()

        # Look up virtual key
        record = await self.db.fetch_one(
            "SELECT * FROM byok_virtual_keys WHERE key_hash = $1 AND active = true",
            key_hash,
        )

        if not record:
            return {"allowed": False, "reason": "invalid_key"}

        # Check model allowlist
        allowed_models = json.loads(record["models"])
        if model not in allowed_models and "*" not in allowed_models:
            return {"allowed": False, "reason": "model_not_allowed",
                    "allowed_models": allowed_models}

        # Check rate limits (Redis)
        rpm_key = f"byok:rpm:{record['key_id']}"
        current_rpm = await self.redis.incr(rpm_key)
        if current_rpm == 1:
            await self.redis.expire(rpm_key, 60)
        if current_rpm > record["rpm"]:
            return {"allowed": False, "reason": "rpm_exceeded",
                    "limit": record["rpm"]}

        # Check monthly budget
        month_key = f"byok:budget:{record['key_id']}:{datetime.now(timezone.utc).strftime('%Y-%m')}"
        spent = float(await self.redis.get(month_key) or 0)
        if spent >= record["monthly_budget_usd"]:
            return {"allowed": False, "reason": "budget_exceeded",
                    "spent": spent, "limit": record["monthly_budget_usd"]}

        return {
            "allowed": True,
            "tenant_id": record["tenant_id"],
            "team_id": record["team_id"],
            "key_id": record["key_id"],
            "remaining_budget": record["monthly_budget_usd"] - spent,
        }

    async def route_to_provider(self, tenant_id: str, user_id: str,
                                provider: str, model: str,
                                prompt: str, max_tokens: int) -> dict:
        """Route call to provider using customer's BYOK key."""
        # Retrieve key from vault (never from database)
        secret = await self.vault.read(
            f"byok/{tenant_id}/{user_id}/{provider}")
        provider_key = secret["api_key"]

        # Call provider directly with customer's key
        # In production: use httpx to call provider API
        response = await self._call_provider(
            provider=provider, api_key=provider_key,
            model=model, prompt=prompt,
            max_tokens=max_tokens,
        )

        # Track cost against virtual key budget
        cost = self._calculate_cost(provider, model,
                                    response["input_tokens"],
                                    response["output_tokens"])

        await self.redis.incrbyfloat(
            f"byok:budget:{tenant_id}:{datetime.now(timezone.utc).strftime('%Y-%m')}",
            cost,
        )

        return {
            "response": response["text"],
            "model": model,
            "provider": provider,
            "input_tokens": response["input_tokens"],
            "output_tokens": response["output_tokens"],
            "cost": cost,
            "billed_to": "customer_provider_account",
        }

    async def revoke_virtual_key(self, tenant_id: str,
                                 key_id: str) -> bool:
        """Revoke a virtual key (5-second revocation)."""
        await self.db.execute(
            "UPDATE byok_virtual_keys SET active = false WHERE key_id = $1 AND tenant_id = $2",
            key_id, tenant_id,
        )

        await self.audit.log(
            tenant_id=tenant_id,
            action="byok_virtual_key_revoked",
            entity_type="virtual_key",
            after={"key_id": key_id},
        )

        return True

    async def rotate_provider_key(self, tenant_id: str, user_id: str,
                                  provider: str, new_key: str) -> str:
        """Rotate customer's provider key with zero downtime."""
        # Write new key to vault
        await self.vault.write(
            f"byok/{tenant_id}/{user_id}/{provider}",
            api_key=new_key,
        )

        await self.audit.log(
            tenant_id=tenant_id, user_id=user_id,
            action="byok_key_rotated",
            entity_type="provider_key",
            after={"provider": provider},
        )

        return "rotated"

    def _calculate_cost(self, provider: str, model: str,
                        input_tokens: int, output_tokens: int) -> float:
        """Calculate cost based on provider pricing."""
        pricing = {
            "openai": {"gpt-4o": {"input": 2.50, "output": 10.00},
                       "gpt-4o-mini": {"input": 0.15, "output": 0.60}},
            "anthropic": {"claude-3-sonnet": {"input": 3.00, "output": 15.00},
                          "claude-3-haiku": {"input": 0.25, "output": 1.25}},
        }

        p = pricing.get(provider, {}).get(model, {"input": 0, "output": 0})
        cost = ((input_tokens / 1_000_000) * p["input"] +
                (output_tokens / 1_000_000) * p["output"])
        return round(cost, 6)

    async def _call_provider(self, provider: str, api_key: str,
                             model: str, prompt: str,
                             max_tokens: int) -> dict:
        """Call LLM provider (simplified)."""
        # In production: use httpx to call provider API
        return {"text": "response", "input_tokens": 100, "output_tokens": 50}

BYOK Cost Savings Checklist

• [ ] Audit current LLM spend and identify token markup costs
• [ ] Check if enterprise has direct provider contracts (64% of enterprises do)
• [ ] Compare BYOK savings vs managed aggregator markup (5% vs 0%)
• [ ] Calculate break-even: BYOK saves when spend exceeds $500-1K/month
• [ ] Choose architecture pattern: gateway (observability), embedded (compliance), hybrid
• [ ] Implement centralized key management with vault (HashiCorp Vault, AWS Secrets Manager)
• [ ] Store provider keys encrypted at rest (AES-256)
• [ ] Never store provider keys in database — use vault only
• [ ] Never log API key values in any system
• [ ] Implement virtual API keys in front of provider credentials
• [ ] Virtual keys: revocable in 5 seconds, not 5-hour key rotation
• [ ] Set model allowlists per virtual key (prevent accidental frontier model calls)
• [ ] Set RPM and TPM rate limits per virtual key
• [ ] Set monthly USD budget caps per virtual key
• [ ] Tag virtual keys by team for spend attribution
• [ ] Implement key rotation with zero downtime
• [ ] Implement key revocation (one-click)
• [ ] Log all key lifecycle events (provision, revoke, rotate) in audit log
• [ ] Choose pricing model: pure passthrough, percentage markup, or flat per-call
• [ ] If percentage markup: justify with platform value (orchestration, observability, agents)
• [ ] If pure passthrough: charge flat platform fee per seat/workspace/feature
• [ ] Use vendor Admin APIs for scoped per-developer key provisioning (KeyGate pattern)
• [ ] Support multiple providers (OpenAI, Anthropic, Azure, Bedrock, Google Vertex)
• [ ] Implement fallback chain across providers
• [ ] Preserve enterprise volume discounts via direct provider contracts
• [ ] Preserve prompt caching commitments from direct contracts
• [ ] Preserve enterprise SLAs from direct contracts
• [ ] Keep aggregator account for experimentation, BYOK for production
• [ ] Implement per-tenant key isolation for multi-tenant platforms
• [ ] Integrate with BYOK multi-tenant architecture
• [ ] Calculate enterprise AI TCO with BYOK savings
• [ ] Track AI usage per virtual key
• [ ] Apply LLM cost optimization on top of BYOK
• [ ] Set AI budget caps per virtual key
• [ ] Implement LLM usage metering for BYOK billing
• [ ] Track per-tenant cost with BYOK
• [ ] Log all BYOK operations in audit logs
• [ ] Apply RBAC to key management
• [ ] Apply zero-trust to API key access
• [ ] Apply OWASP LLM Top 10 security controls
• [ ] Test: key storage and retrieval from vault
• [ ] Test: virtual key authentication and governance (allowlist, rate limit, budget)
• [ ] Test: key revocation works in under 5 seconds
• [ ] Test: key rotation with zero downtime
• [ ] Test: fallback chain across providers
• [ ] Test: cost tracking matches provider billing
• [ ] Document BYOK architecture, pricing model, and key governance

FAQ

What is BYOK and how does it save costs?

BYOK (Bring Your Own Key) lets customers use their own LLM provider API keys instead of paying the platform's markup on tokens. osFoundry: "A 2025 Gartner snapshot put 64% of enterprise GenAI buyers as already holding direct provider contracts. Re-billing them through your account is friction. BYOK lets the customer absorb token volatility directly while you charge a flat platform fee." VerticalAPI: "BYOK gateways charge 0% token markup vs managed aggregators at ~5% on top of provider list. At $5K/month spend that is $250/month savings, often more than the BYOK platform fee." Cost savings: (1) No token markup — customer pays provider directly. (2) Volume discounts — enterprise negotiated rates preserved. (3) Prompt caching commitments — direct provider contracts include caching discounts. (4) Enterprise SLAs — direct contracts include uptime guarantees. (5) No re-billing friction — enterprise buyers use committed budget.

What are the BYOK architecture patterns?

Three BYOK architecture patterns exist, each with different cost, latency, and trust implications. osFoundry: (1) Centralized gateway — all LLM calls route through your server using customer keys stored encrypted. Best for centralized observability and fallback orchestration. (2) Embedded SDK pass-through — actual model call originates client-side, server issues short-lived signed tokens. Best for regulated customers who cannot route data through a third party. (3) Hybrid — centralized key management but direct client-to-provider calls. Best when you need key management without data routing through your server. KeyGate: "No proxy, zero added latency. KeyGate is only involved at key lifecycle time. Uses vendor Admin APIs to create scoped projects/workspaces per developer and issue real API keys within them." Choose: gateway for observability, embedded for compliance, hybrid for balance.

How do BYOK products price and charge customers?

BYOK products use three pricing models. osFoundry: (1) Pure passthrough — charge zero on tokens (billed to customer provider account), charge flat platform fee per seat/workspace/feature. Cleanest legally, hardest to grow ARPU. (2) Percentage markup — charge X% on top of token cost as platform fee. osFoundry uses 7.5%. Revenue scales with usage, aligning incentives with customer value. Risk: customers ask why they pay markup when it is their key. (3) Flat per-call pricing — fixed fee per API call regardless of token count. Kinde: "BYOK pricing reduces financial risk for you and provides cost transparency for customers, making it particularly popular for AI and machine learning-based SaaS products." VerticalAPI: "Choose BYOK gateway when monthly LLM spend exceeds roughly $500-1,000 and the 5% aggregator markup starts to matter."

How do you manage API keys securely in a BYOK system?

Manage API keys securely using virtual API keys and centralized key lifecycle management. Gateway-LLM: "A virtual API key is a short, revocable token your application uses to call the gateway. It sits in front of your provider credentials, so a leaked key is a five-second revocation, not a five-hour key rotation. Each one carries a rate limit (RPM and TPM), a model allowlist, and an optional monthly USD budget." Alloy: "Replace per-developer API keys with sk-alloy-... prefixed keys. Revoke, rotate or audit any key from a single pane. No more shared credentials buried in CI pipelines." KeyGate: "Per-developer scoping — each key lives in its own vendor project/workspace. Budget controls at vendor level. Rate limiting per-key RPM. One-click rotation. Audit log for every provision, revoke, and config change." Security: (1) Encrypt keys at rest (AES-256). (2) Never log key values. (3) Use virtual keys in front of provider keys. (4) Scope keys per team/project. (5) Set model allowlists. (6) Set monthly budget caps. (7) Rotate regularly.

When should you choose BYOK vs managed LLM providers?

Choose BYOK when monthly LLM spend exceeds $500-1,000 and you have direct provider contracts. VerticalAPI: "BYOK gateways save you the 5% markup without sacrificing observability. At $5K/month spend that is $250/month savings. BYOK preserves your direct provider contracts — volume discounts, prompt caching commitments, enterprise SLAs. The trade-off is signing up with each provider once (10-20 minutes per provider)." Choose managed aggregators (e.g., OpenRouter) when: (1) Spend is below $500/month. (2) Quick prototyping. (3) Simple billing preferred. (4) No need for direct provider contracts. VerticalAPI: "The architectures are not mutually exclusive — many teams keep an aggregator account for experimentation and use BYOK for production traffic." JetBrains: "BYOK makes it easy to bring frontier models, cost-efficient small models, locally hosted private models, or experimental research previews directly into your IDE — with no subscription required." Augment Code: "Teams planning production deployments need an architecture that avoids credential sprawl, supports chargeback, and keeps multi-provider routing configurable."

Want a self-hosted AI company brain that does all of this out of the box?
Book a demo →