OpenRouter vs LiteLLM: Managed vs Self-Hosted LLM Gateway Comparison

By Fadi Updated Feb 8, 2026 13 min read LLM Gateways

TL;DR — OpenRouter vs LiteLLM: managed vs self-hosted. APIScout: "OpenRouter for managed 500+ models, no infrastructure. LiteLLM for self-hosted control, zero markup, enterprise RBAC. Most startups start with OpenRouter, most enterprises end up on LiteLLM. 5% markup = $50K/year on $1M spend." OpenRouter Blog: "Both OpenAI-compatible. OpenRouter runs layer for you. LiteLLM runs in your infra — data stays on network, no platform fee, but you operate PostgreSQL, Redis, Docker. Not mutually exclusive — LiteLLM can use OpenRouter as upstream." TrueFoundry: "Control vs convenience. LiteLLM self-hosted with GitOps. OpenRouter managed edge SaaS with unified billing." Markaicode: "OpenRouter 4x faster setup. LiteLLM p95 193ms on cache hits, 72% improvement. Both OpenAI-compatible — migration is one-line config change." TechJack: "341 text models on OpenRouter. LiteLLM 100+ providers with virtual keys, budgets, guardrails. No platform team — choose hosted. Data privacy — choose self-hosted." Learn more with LLM gateway, LiteLLM tutorial, OpenRouter tutorial, and switch provider.

APIScout provides the executive summary: "OpenRouter is the default choice for teams that want managed access to 500+ models with no infrastructure overhead. LiteLLM is the right choice for teams that need self-hosted control, zero markup (OpenRouter's 5% = $60K/year at $1M spend), enterprise RBAC, or strict data compliance requirements. Most startups should start with OpenRouter. Most enterprises end up on LiteLLM."

OpenRouter Blog states the core difference: "Both give you a single OpenAI-compatible API across many providers. OpenRouter runs that layer for you, so there's no infrastructure to operate. LiteLLM runs inside your own infrastructure, so your data stays on your network and you pay no platform fee, in exchange for operating PostgreSQL, Redis, and Docker yourself."

OpenRouter vs LiteLLM Architecture

flowchart TD subgraph OpenRouterPath["OpenRouter (Managed SaaS)"] ORApp["Your App
OpenAI SDK
one API key"] ORGateway["OpenRouter Gateway
Cloudflare edge
Auto Router by NotDiamond
automatic failover
500+ models, 60+ providers"] ORProviders["Providers
OpenAI, Anthropic, Google
AWS Bedrock, Mistral, 60+ others
free models available"] end subgraph LiteLLMPath["LiteLLM (Self-Hosted)"] LLApp["Your App
OpenAI SDK
one API key"] LLGateway["LiteLLM Proxy
Docker/K8s in your VPC
6 routing modes
custom Python logic
100+ providers"] LLInfra["Your Infrastructure
PostgreSQL: spend & keys
Redis: cache & rate limits
Docker: proxy container"] LLProviders["Providers
OpenAI, Anthropic, Google
Ollama, vLLM, TGI
100+ providers"] end subgraph Combined["Combined (Best of Both)"] ComboApp["Your App"] ComboLiteLLM["LiteLLM Proxy
local RBAC, budgets
logging, caching"] ComboOR["OpenRouter
as upstream provider
500+ models, failover"] ComboDirect["Direct Providers
OpenAI, Anthropic
Google, local models"] end ORApp --> ORGateway --> ORProviders LLApp --> LLGateway --> LLInfra --> LLProviders ComboApp --> ComboLiteLLM ComboLiteLLM --> ComboOR ComboLiteLLM --> ComboDirect

Feature Comparison

Feature	OpenRouter	LiteLLM
Deployment	Managed SaaS	Self-hosted (Docker/K8s)
Setup time	5 min (API key)	~45 min (Docker + config)
Model catalog	500+ (341 text, 32 image, 26 embeddings)	100+ providers
Markup	5.5% on credits, 5% with BYOK	0% (free open source)
Free models	Yes (DeepSeek R1, Llama 3.3, Gemma 3)	No
Auto-fallback	Yes (automatic, transparent)	Yes (configurable fallback chains)
Routing modes	Auto Router (NotDiamond), provider filtering	6 modes: weighted, latency, rate-limit, least-busy, lowest-cost, custom Python
Virtual keys	Basic	Full-featured with per-key budgets
Per-team budgets	Basic	Comprehensive (YAML config)
SSO/SAML	No	Enterprise only
RBAC	No	Enterprise only
Data location	OpenRouter infra (Cloudflare edge)	Your infra (VPC)
Observability	Basic	Pluggable (Langfuse, Helicone, MLflow, OpenTelemetry)
Caching	No (provider-side only)	Redis-backed with configurable TTL
GPU/local models	No	Yes (Ollama, vLLM, TGI)
Open source	No	Yes (Apache 2.0)
Throughput	180 req/s (managed)	120 req/s (self-hosted G4dn)
p95 latency	Provider-dependent	193ms on cache hits

Pricing Comparison

Cost Factor	OpenRouter	LiteLLM
Platform fee	5.5% on credits, 5% with BYOK	0%
First 1M requests	Free each month	N/A (you pay provider directly)
Failed requests	Not billed	N/A
Infrastructure	$0 (managed)	~$200-500/month (Postgres, Redis, compute)
$1M AI spend	~$50K/year in fees	~$0 in fees + infra costs
Enterprise features	N/A	SSO, RBAC, audit logs (sales pricing)

Implementation

from dataclasses import dataclass
from typing import Optional
from enum import Enum

class GatewayChoice(Enum):
    OPENROUTER = "openrouter"
    LITELLM = "litellm"
    COMBINED = "combined"

@dataclass
class OpenRouterVsLiteLLMGuide:
    """OpenRouter vs LiteLLM comparison guide."""

    def get_openrouter_overview(self) -> dict:
        """OpenRouter overview."""
        return {
            "type": "Managed SaaS gateway",
            "infrastructure": "Cloudflare edge, no servers to run",
            "models": "500+ (341 text, 32 image, 26 embeddings, 14 video, 10 transcription, 9 speech)",
            "providers": "60+ providers",
            "endpoint": "https://openrouter.ai/api/v1",
            "setup": "5 minutes — copy API key, point OpenAI SDK",
            "pricing": (
                "5.5% platform fee on pay-as-you-go credits, "
                "5% with Bring Your Own Key, "
                "$0.80 minimum per purchase, "
                "first 1M requests free each month, "
                "failed requests not billed, "
                "free models available"
            ),
            "routing": (
                "Auto Router powered by NotDiamond picks model per prompt. "
                "Provider-level routing deprioritizes providers with "
                "outages in last 30 seconds. "
                "Filter by price, throughput, latency, data policy, "
                "ZDR, quantization."
            ),
            "fallback": (
                "Automatic — falls back to next available provider "
                "for same model family, transparently to application. "
                "No circuit breaker logic needed."
            ),
            "free_models": [
                "DeepSeek R1",
                "Llama 3.3 70B",
                "Gemma 3",
            ],
            "best_for": (
                "Startups, teams without DevOps, "
                "want instant setup, need 500+ models, "
                "want free models, prefer pay-per-use"
            ),
        }

    def get_litellm_overview(self) -> dict:
        """LiteLLM overview."""
        return {
            "type": "Open-source self-hosted proxy",
            "infrastructure": (
                "Docker container or K8s pod in your VPC. "
                "PostgreSQL for spend data and keys. "
                "Redis for caching and rate limits."
            ),
            "models": "100+ LLM providers",
            "providers": "OpenAI, Anthropic, Google, AWS Bedrock, "
                         "Azure, Ollama, vLLM, TGI, 100+ more",
            "endpoint": "http://localhost:4000/v1 (your infra)",
            "setup": "~45 minutes — Docker + config.yaml",
            "pricing": (
                "Free open source (Apache 2.0). "
                "Pay provider costs directly. "
                "Infrastructure ~$200-500/month for production. "
                "Enterprise: SSO, RBAC, audit logs (sales pricing). "
                "Maintained by BerriAI."
            ),
            "routing_modes": [
                "Weighted pick",
                "Latency-based",
                "Rate-limit-aware",
                "Least-busy",
                "Lowest-cost",
                "Custom Python logic",
            ],
            "fallback": (
                "Configurable fallback lists — "
                "try next model when one fails. "
                "Per-team and per-model rules "
                "enforced at proxy level."
            ),
            "enterprise_features": [
                "Virtual keys with per-key budgets",
                "Per-team budget enforcement",
                "RBAC and SSO/SAML",
                "Audit logs",
                "Prometheus metrics",
                "Guardrails (content filtering, PII masking)",
                "Admin dashboard",
            ],
            "caching": (
                "Redis-backed caching with configurable TTL. "
                "p95 latency 193ms on warm cache hits. "
                "72% improvement over external provider."
            ),
            "gpu_support": (
                "Route to local models: Ollama, vLLM, TGI. "
                "Direct GPU integration."
            ),
            "best_for": (
                "Enterprises, teams with DevOps, "
                "data privacy requirements, "
                "want zero markup, need enterprise RBAC/SSO, "
                "want custom routing, need GPU/local model support"
            ),
        }

    def get_combined_approach(self) -> dict:
        """Using LiteLLM with OpenRouter as upstream."""
        return {
            "concept": (
                "LiteLLM can use OpenRouter as an upstream "
                "provider, combining local control with "
                "managed breadth"
            ),
            "benefits": [
                "LiteLLM local RBAC and budget enforcement",
                "LiteLLM logging and observability",
                "LiteLLM caching with Redis",
                "OpenRouter 500+ model access",
                "OpenRouter automatic multi-provider failover",
                "Direct provider connections for critical models",
            ],
            "config": (
                "# LiteLLM config.yaml\n"
                "model_list:\n"
                "  - model_name: smart\n"
                "    litellm_params:\n"
                "      model: openrouter/"
                "        anthropic/claude-3.5-sonnet\n"
                "      api_key: os.environ/"
                "        OPENROUTER_API_KEY\n"
                "  - model_name: smart\n"
                "    litellm_params:\n"
                "      model: anthropic/"
                "        claude-3.5-sonnet\n"
                "      api_key: os.environ/"
                "        ANTHROPIC_API_KEY\n"
                "  - model_name: cheap\n"
                "    litellm_params:\n"
                "      model: openrouter/"
                "        google/gemini-2.0-flash\n"
                "      api_key: os.environ/"
                "        OPENROUTER_API_KEY\n"
                "\n"
                "router_settings:\n"
                "  routing_strategy: lowest-cost\n"
                "  fallbacks:\n"
                "    - smart: [cheap]"
            ),
        }

    def get_decision_matrix(self) -> dict:
        """Decision matrix."""
        return {
            "choose_openrouter": {
                "when": [
                    "No DevOps or platform team",
                    "Want 4x faster setup (5 min vs 45 min)",
                    "Need 500+ model access",
                    "Want free models (DeepSeek, Llama, Gemma)",
                    "Prefer pay-per-use, no infrastructure",
                    "Want automatic failover without coding",
                    "Startup or small team",
                ],
            },
            "choose_litellm": {
                "when": [
                    "Data privacy/sovereignty requirements",
                    "Have DevOps capacity",
                    "Want zero markup (save $50K+/year at scale)",
                    "Need enterprise RBAC, SSO, audit logs",
                    "Want custom routing logic in Python",
                    "Need GPU/local model support (Ollama, vLLM)",
                    "Want Redis-backed caching",
                    "Enterprise or regulated industry",
                    "Want GitOps workflows for config",
                ],
            },
            "choose_combined": {
                "when": [
                    "Want local control AND managed breadth",
                    "Need RBAC + 500+ models",
                    "Want caching + automatic failover",
                    "Hybrid: critical models direct, "
                    "rest via OpenRouter",
                ],
            },
        }

    def get_code_examples(self) -> dict:
        """Code examples for both."""
        return {
            "openrouter": (
                "from openai import OpenAI\n"
                "\n"
                "client = OpenAI(\n"
                "    base_url='https://openrouter.ai'\n"
                "        '/api/v1',\n"
                "    api_key='sk-or-v1-xxx',\n"
                ")\n"
                "\n"
                "response = client.chat.completions\\\n"
                "    .create(\n"
                "    model='anthropic/claude-3.5'\n"
                "        '-sonnet',\n"
                "    messages=[\n"
                "        {'role': 'user',\n"
                "         'content': 'Hello!'}\n"
                "    ],\n"
                ")"
            ),
            "litellm": (
                "from openai import OpenAI\n"
                "\n"
                "client = OpenAI(\n"
                "    base_url='http://localhost:'\n"
                "        '4000/v1',\n"
                "    api_key='sk-litellm-xxx',\n"
                ")\n"
                "\n"
                "response = client.chat.completions\\\n"
                "    .create(\n"
                "    model='smart',  # LiteLLM routes\n"
                "    messages=[\n"
                "        {'role': 'user',\n"
                "         'content': 'Hello!'}\n"
                "    ],\n"
                ")"
            ),
            "litellm_sdk": (
                "import litellm\n"
                "\n"
                "response = litellm.completion(\n"
                "    model='anthropic/claude-3.5'\n"
                "        '-sonnet',\n"
                "    messages=[\n"
                "        {'role': 'user',\n"
                "         'content': 'Hello!'}\n"
                "    ],\n"
                "    api_key='sk-ant-xxx',\n"
                ")\n"
                "print(response.choices[0]\n"
                "      .message.content)"
            ),
        }

OpenRouter vs LiteLLM Checklist

[ ] OpenRouter: managed SaaS gateway, no infrastructure, 500+ models, 60+ providers
[ ] OpenRouter: 5 min setup, Cloudflare edge, OpenAI-compatible API
[ ] OpenRouter: 5.5% platform fee on credits, 5% with BYOK, first 1M requests free
[ ] OpenRouter: free models — DeepSeek R1, Llama 3.3 70B, Gemma 3
[ ] OpenRouter: Auto Router by NotDiamond picks model per prompt
[ ] OpenRouter: automatic failover — falls back to next provider transparently
[ ] OpenRouter: filter routing by price, throughput, latency, data policy, ZDR, quantization
[ ] OpenRouter: 341 text, 32 image, 26 embeddings, 14 video, 10 transcription, 9 speech models
[ ] OpenRouter: failed requests not billed
[ ] OpenRouter: no SSO, no RBAC, basic virtual keys, basic observability
[ ] LiteLLM: open-source self-hosted proxy, Apache 2.0, maintained by BerriAI
[ ] LiteLLM: Docker/K8s in your VPC, PostgreSQL + Redis + Docker
[ ] LiteLLM: ~45 min setup, 100+ providers, OpenAI-compatible endpoint
[ ] LiteLLM: free open source, 0% markup, pay provider directly, infra ~$200-500/month
[ ] LiteLLM: 6 routing modes — weighted, latency, rate-limit, least-busy, lowest-cost, custom Python
[ ] LiteLLM: configurable fallback chains — try next model when one fails
[ ] LiteLLM: full-featured virtual keys with per-key, per-team, per-user budgets
[ ] LiteLLM: enterprise features — SSO/SAML, RBAC, audit logs, Prometheus metrics
[ ] LiteLLM: guardrails — content filtering, PII masking
[ ] LiteLLM: Redis-backed caching with configurable TTL, p95 193ms on cache hits
[ ] LiteLLM: GPU/local model support — Ollama, vLLM, TGI
[ ] LiteLLM: pluggable observability — Langfuse, Helicone, MLflow, OpenTelemetry
[ ] LiteLLM: GitOps workflows — config in YAML under version control
[ ] LiteLLM: admin dashboard for management
[ ] Combined: LiteLLM can use OpenRouter as upstream provider
[ ] Combined: local RBAC + logging + caching + 500+ model access + automatic failover
[ ] Combined: critical models direct, rest via OpenRouter
[ ] Both: OpenAI-compatible API — migration is one-line config change
[ ] Both: automatic fallback on provider failure
[ ] Both: unified API across multiple providers
[ ] Choose OpenRouter: no DevOps team, want instant setup, need 500+ models, want free models
[ ] Choose OpenRouter: prefer pay-per-use, startup or small team, want automatic failover
[ ] Choose LiteLLM: data privacy required, have DevOps, want zero markup, need enterprise RBAC/SSO
[ ] Choose LiteLLM: want custom routing, need GPU/local models, enterprise or regulated industry
[ ] Choose combined: want local control AND managed breadth, hybrid approach
[ ] OpenRouter 5% markup = $50K/year on $1M AI spend — LiteLLM saves this at scale
[ ] OpenRouter throughput: 180 req/s (managed), LiteLLM: 120 req/s (self-hosted G4dn)
[ ] OpenRouter can be 15-25% cheaper than direct provider rates via bulk negotiation
[ ] LiteLLM caching: 72% latency improvement on warm cache hits
[ ] Most startups start with OpenRouter, most enterprises end up on LiteLLM
[ ] Read LLM gateway guide for gateway concepts
[ ] Read LiteLLM tutorial for self-hosted setup
[ ] Read OpenRouter tutorial for managed setup
[ ] Read switch LLM provider for provider migration
[ ] Test: OpenRouter API key works with OpenAI SDK
[ ] Test: LiteLLM proxy routes to correct provider
[ ] Test: fallback switches on provider failure for both
[ ] Test: LiteLLM caching reduces latency on repeated requests
[ ] Test: LiteLLM budget enforcement blocks over-spend
[ ] Test: combined setup — LiteLLM routes to OpenRouter upstream
[ ] Document choice rationale, pricing impact, routing config, and fallback strategy

FAQ

What is the difference between OpenRouter and LiteLLM?

OpenRouter is a managed SaaS gateway with 500+ models and no infrastructure. LiteLLM is an open-source self-hosted proxy with 100+ providers and maximum control. APIScout: "OpenRouter is the default choice for teams that want managed access to 500+ models with no infrastructure overhead. LiteLLM is the right choice for teams that need self-hosted control, zero markup, enterprise RBAC, or strict data compliance. Most startups should start with OpenRouter. Most enterprises end up on LiteLLM." OpenRouter Blog: "Both give you a single OpenAI-compatible API across many providers. OpenRouter runs that layer for you, no infrastructure to operate. LiteLLM runs inside your own infrastructure, data stays on your network, you pay no platform fee, in exchange for operating PostgreSQL, Redis, and Docker yourself." TrueFoundry: "LiteLLM vs OpenRouter comes down to control vs convenience: LiteLLM is self-hosted open-source proxy you run and govern, while OpenRouter is managed cloud aggregator with instant multi-model access and unified billing."

How do OpenRouter and LiteLLM compare on pricing?

OpenRouter charges 0-5.5% markup on pay-as-you-go credits. LiteLLM is free to self-host but you pay for infrastructure. APIScout: "OpenRouter 5% markup means $50,000/year in gateway fees on $1M AI spend. For high-volume teams, LiteLLM zero-markup self-hosted model pays for itself quickly." OpenRouter Blog: "OpenRouter passes provider pricing through at 0% markup, then charges 5.5% platform fee on pay-as-you-go credit purchases, with $0.80 minimum. Bring Your Own Key drops fee to 5%, first 1M requests each month waived. Failed requests not billed. LiteLLM is free to self-host. You pay for infrastructure: PostgreSQL, Redis, compute — typically few hundred dollars/month for production." Markaicode: "OpenRouter pay-per-token pricing can be 15-25% cheaper than direct provider rates thanks to bulk negotiation. LiteLLM built-in caching delivers p95 latencies below 200ms on warm cache hits." Pricing: OpenRouter — 5.5% fee on credits, 5% with BYOK, first 1M requests free, free models available. LiteLLM — free open source, pay infrastructure (~$200-500/month), Enterprise adds SSO/RBAC/audit logs (sales pricing).

Which has better routing and fallback: OpenRouter or LiteLLM?

Both offer routing and fallback but with different approaches. OpenRouter has Auto Router powered by NotDiamond and automatic provider-level failover. LiteLLM has six routing modes and custom Python logic. OpenRouter Blog: "OpenRouter Auto Router, powered by NotDiamond, picks a model per prompt. Provider-level routing deprioritizes any provider with outages in last 30 seconds. Constrain routing with provider object filtering by price, throughput, latency, data policy, ZDR, quantization. LiteLLM gives more strategies and full custom logic. Six routing modes: weighted pick, latency-based, rate-limit-aware, least-busy, lowest-cost, custom Python. Fallback lists try next model when one fails." APIScout: "OpenRouter automatically falls back to next available provider for same model family — transparently to your application. No circuit breaker logic or retry strategies needed." TrueFoundry: "LiteLLM: native weighted load balancing and configurable fallback chains. OpenRouter: intelligent provider routing with automatic failover when providers unavailable."

Can you use LiteLLM with OpenRouter together?

Yes, LiteLLM can use OpenRouter as an upstream provider, combining LiteLLM local control with OpenRouter multi-provider breadth. OpenRouter Blog: "The two are not mutually exclusive. LiteLLM can use OpenRouter as an upstream provider, so you get LiteLLM local RBAC and logging while OpenRouter handles multi-provider failover and model breadth." TechJack: "LiteLLM can route to OpenRouter as one of its providers, combining self-hosted control with managed breadth." Use case: LiteLLM as your local proxy with RBAC, budget enforcement, logging, and caching. OpenRouter as one upstream provider for 500+ model access and automatic failover. Other upstream providers configured directly (OpenAI, Anthropic, Google). Best of both worlds: local control + managed breadth.

When should you choose OpenRouter over LiteLLM?

Choose OpenRouter for simplicity, no infrastructure, fast setup, and free models. Choose LiteLLM for control, data sovereignty, zero markup, and enterprise features. APIScout: "Most startups should start with OpenRouter. Most enterprises end up on LiteLLM." Markaicode: "Pick OpenRouter when you need 4x faster setup — no server to provision or maintain. LiteLLM lets you enforce data sovereignty and inspect every request payload." TrueFoundry: "OpenRouter for rapid setup and turnkey routing without infrastructure management. LiteLLM for complete ownership of inference stack with GitOps workflows." TechJack: "If you do not have a platform or infrastructure team, the calculus tilts hard toward hosted. Self-hosted LiteLLM is the safer default for data privacy." Choose OpenRouter when: no DevOps team, want instant setup, need 500+ models, want free models, prefer pay-per-use. Choose LiteLLM when: data privacy required, have DevOps capacity, want zero markup, need enterprise RBAC/SSO, want custom routing logic, need GPU/local model support.

Want a self-hosted AI company brain that does all of this out of the box?
Book a demo →