How to Estimate LLM Costs Before You Build: Token Pricing and Budget Forecasting
TL;DR — LLM cost estimation: six variables, four layers, big multipliers. SumatoSoft: "Monthly Cost = ((Users x DAU x Sessions x Input_Tokens x Input_Price) + (Output_Tokens x Output_Price) + (Retry x Base) - (Cache x Cacheable)) x Agent_Multiplier. Agent multiplier: 1x simple RAG, 5-10x agentic." AICost.ai: "Output tokens cost 3-5x input at every vendor. Cache saves 90%, batch saves 50%." CreatorOpsMatrix: "Four cost layers: LLM API, vector DB, automation, monitoring. $868/mo total — $833 LLM, $25 vector, $10 automation." Gekro: "Self-hosting competitive at tens to hundreds of millions of tokens/month. Under-loaded GPU costs more per token than API." Learn more with choose right LLM, RAG vs fine-tuning, FastAPI tutorial, and monitoring.
SumatoSoft provides the master formula: "Monthly Cost = ((Users x DAU x Sessions/day x Avg_Input_Tokens x Input_Price) + (Users x DAU x Sessions/day x Avg_Output_Tokens x Output_Price) + (Retry_rate x Base_cost) - (Cache_hit_rate x Cacheable_cost)) x Agent_Multiplier. The formula has six variables: active users, sessions, input and output tokens per session, retry rate, cache hit rate, and agent multiplier."
AICost.ai simplifies: "Cost = (input + output tokens) x the model's rate x requests x ~30 days. Output tokens cost 3-5x more than input tokens at every major vendor."
LLM Cost Estimation Architecture
daily active users
sessions per user
messages per session"] Tokens["Token Counts
input: prompt + context + history
output: response (constrained)
output costs 3-5x input"] Price["Token Pricing
input price per 1M tokens
output price per 1M tokens
varies by model tier"] end subgraph Adjustments["Cost Adjustments"] Cache["Prompt Caching
80-90% off cached input
cache hit rate 40-60%
saves above 22% hit rate"] Batch["Batch API
50% off all token costs
non-real-time tasks
results within 24h"] Retry["Retry Overhead
5-15% buffer for failures
rate limits, timeouts
model refusals"] Agent["Agent Multiplier
simple RAG: 1x
self-correcting: 3x
light agent: 3-5x
full agentic: 5-10x"] end subgraph Tiers["Model Tier Routing"] Economy["Economy Tier
GPT-4o-mini: $0.15/$0.60
DeepSeek V3: $0.27/$1.10
Qwen 7B: $20/mo self-hosted"] Workhorse["Workhorse Tier
Claude Sonnet: $3/$15
Gemini Pro: $2/$12
Llama 70B: $45/mo self-hosted"] Frontier["Frontier Tier
GPT-5: $5/$15
Claude Opus: $15/$75
flagship 15-30x economy"] end subgraph Layers["Cost Layers"] LLM["LLM API Cost
biggest variable
80-95% of total
driven by tokens x price"] VectorDB["Vector Database
Qdrant: $20-40/mo
Pinecone: $50+/mo
pgvector: free"] Infra["Infrastructure
automation platform
monitoring
embedding model
API gateway"] Hidden["Hidden Costs
reasoning tokens 5-20x
retry overhead 5-15%
prompt engineering time
MLOps engineering"] end subgraph Decision["Build vs Buy Decision"] API["API Path
under 50M tokens/mo
no MLOps needed
pay per token
instant scaling"] SelfHost["Self-Hosted Path
over 50-100M tokens/mo
GPU + electricity
MLOps engineering
flat monthly cost"] end Users --> Tokens Tokens --> Price Price --> Cache Price --> Batch Price --> Retry Cache --> Agent Batch --> Agent Retry --> Agent Agent --> Economy Agent --> Workhorse Agent --> Frontier Economy --> LLM Workhorse --> LLM Frontier --> LLM LLM --> VectorDB LLM --> Infra LLM --> Hidden LLM --> API LLM --> SelfHost style Inputs fill:#4169E1,color:#fff style Adjustments fill:#39FF14,color:#000 style Tiers fill:#2D1B69,color:#fff style Layers fill:#FF6B6B,color:#fff
Cost Formula Reference
| Variable | Description | Typical Values |
|---|---|---|
| Input tokens | Prompt + context + history | 300-1500 chat, 2K-8K RAG, 8K-100K long-doc |
| Output tokens | Model response | 10-50 classify, 150-600 chat, 500-2000 code |
| Input price | Per 1M input tokens | $0.15 (mini) to $15 (Opus) |
| Output price | Per 1M output tokens | $0.60 (mini) to $75 (Opus) |
| Cache hit rate | Fraction of cached input | 0% (cold) to 60-90% (stable prompts) |
| Batch discount | 50% off for async | 0% (real-time) or 50% (batch) |
| Retry rate | Failed request buffer | 5-15% |
| Agent multiplier | LLM calls per message | 1x (RAG) to 10x (agentic) |
| Days/month | Operating days | 30 (24/7), 22 (business days) |
Implementation
from dataclasses import dataclass
from typing import Optional
from enum import Enum
class ModelTier(Enum):
ECONOMY = "economy"
WORKHORSE = "workhorse"
FRONTIER = "frontier"
@dataclass
class LLMCostEstimator:
"""LLM cost estimation and forecasting guide."""
def get_cost_calculator(self) -> str:
"""LLM cost calculator implementation."""
return (
"# === LLM COST CALCULATOR ===\n"
"\n"
"# Token prices per 1M tokens\n"
"MODEL_PRICING = {\n"
" 'gpt-4o-mini': {\n"
" 'in': 0.15, 'out': 0.60},\n"
" 'gpt-5': {\n"
" 'in': 5.0, 'out': 15.0},\n"
" 'claude-sonnet': {\n"
" 'in': 3.0, 'out': 15.0},\n"
" 'claude-opus': {\n"
" 'in': 15.0, 'out': 75.0},\n"
" 'gemini-pro': {\n"
" 'in': 2.0, 'out': 12.0},\n"
" 'deepseek-v3': {\n"
" 'in': 0.27, 'out': 1.10},\n"
" 'llama-70b-self': {\n"
" 'in': 0, 'out': 0}, # flat\n"
"}\n"
"\n"
"SELFHOSTED_MONTHLY = {\n"
" 'llama-70b': 45, # RTX 4090\n"
" 'qwen-72b': 45,\n"
" 'qwen-32b': 45,\n"
" 'qwen-7b': 20, # A4000\n"
"}\n"
"\n"
"def estimate_monthly_cost(\n"
" model: str,\n"
" daily_users: int,\n"
" sessions_per_user: int,\n"
" input_tokens_per_session: int,\n"
" output_tokens_per_session: int,\n"
" cache_hit_rate: float = 0.0,\n"
" batch_discount: bool = False,\n"
" retry_rate: float = 0.10,\n"
" agent_multiplier: float = 1.0,\n"
" days_per_month: int = 30):\n"
" '''Estimate monthly LLM cost.''' \n"
" pricing = MODEL_PRICING[model]\n"
" \n"
" # Daily token volume\n"
" daily_sessions = (\n"
" daily_users\n"
" * sessions_per_user)\n"
" \n"
" daily_input = (\n"
" daily_sessions\n"
" * input_tokens_per_session\n"
" * agent_multiplier)\n"
" daily_output = (\n"
" daily_sessions\n"
" * output_tokens_per_session\n"
" * agent_multiplier)\n"
" \n"
" # Cache adjustment\n"
" cached_input = (\n"
" daily_input * cache_hit_rate)\n"
" fresh_input = (\n"
" daily_input * (1 - cache_hit_rate))\n"
" \n"
" # Batch discount\n"
" batch_mult = (\n"
" 0.5 if batch_discount else 1.0)\n"
" \n"
" # Monthly cost\n"
" input_cost = (\n"
" fresh_input * days_per_month\n"
" * pricing['in'] / 1_000_000)\n"
" cached_cost = (\n"
" cached_input * days_per_month\n"
" * pricing['in'] * 0.1\n"
" / 1_000_000) # 90% off\n"
" output_cost = (\n"
" daily_output * days_per_month\n"
" * pricing['out'] * batch_mult\n"
" / 1_000_000)\n"
" \n"
" base = (\n"
" input_cost + cached_cost\n"
" + output_cost)\n"
" \n"
" # Retry overhead\n"
" total = base * (1 + retry_rate)\n"
" \n"
" return {\n"
" 'model': model,\n"
" 'monthly_cost': round(\n"
" total, 2),\n"
" 'input_cost': round(\n"
" input_cost, 2),\n"
" 'cached_cost': round(\n"
" cached_cost, 2),\n"
" 'output_cost': round(\n"
" output_cost, 2),\n"
" 'retry_buffer': round(\n"
" base * retry_rate, 2),\n"
" 'daily_tokens': (\n"
" daily_input\n"
" + daily_output),\n"
" }\n"
"\n"
"# Example: 100 users, 5 msg/day\n"
"# 3 LLM calls/msg (agent=3x)\n"
"# 2000 input, 500 output\n"
"# cache 40%, batch off\n"
"result = estimate_monthly_cost(\n"
" model='claude-sonnet',\n"
" daily_users=100,\n"
" sessions_per_user=5,\n"
" input_tokens_per_session=2000,\n"
" output_tokens_per_session=500,\n"
" cache_hit_rate=0.40,\n"
" agent_multiplier=3.0)\n"
"# Result: ~$833/mo"
)
def get_breakeven_calc(self) -> str:
"""Self-hosted vs API break-even."""
return (
"# === BREAK-EVEN ANALYSIS ===\n"
"\n"
"def breakeven_analysis(\n"
" api_model: str,\n"
" selfhost_model: str,\n"
" monthly_tokens_millions: float):\n"
" '''Compare API vs self-hosted.''' \n"
" # API monthly cost\n"
" api_cost = (\n"
" monthly_tokens_millions * 0.3\n"
" * MODEL_PRICING[api_model]\n"
" ['in']\n"
" + monthly_tokens_millions * 0.7\n"
" * MODEL_PRICING[api_model]\n"
" ['out'])\n"
" \n"
" # Self-hosted flat cost\n"
" self_cost = SELFHOSTED_MONTHLY[\n"
" selfhost_model]\n"
" \n"
" crossover = None\n"
" if api_cost > 0:\n"
" # Find crossover point\n"
" for m in range(1, 200):\n"
" api_m = (\n"
" m * 0.3\n"
" * MODEL_PRICING[api_model]\n"
" ['in']\n"
" + m * 0.7\n"
" * MODEL_PRICING[api_model]\n"
" ['out'])\n"
" if api_m > self_cost:\n"
" crossover = m\n"
" break\n"
" \n"
" return {\n"
" 'api_monthly': round(\n"
" api_cost, 2),\n"
" 'selfhost_monthly': self_cost,\n"
" 'crossover_millions': crossover,\n"
" 'recommendation': (\n"
" 'API' if api_cost\n"
" < self_cost\n"
" else 'self-host'),\n"
" }\n"
"\n"
"# Example: GPT-4o vs Llama 70B\n"
"# At 10M tokens/mo:\n"
"# GPT-4o API: ~$875/mo\n"
"# Llama 70B self: $45/mo\n"
"# Crossover: ~2-3M tokens/mo"
)
def get_rag_cost(self) -> str:
"""RAG pipeline cost breakdown."""
return (
"# === RAG COST BREAKDOWN ===\n"
"\n"
"def rag_monthly_cost(\n"
" queries_per_day: int,\n"
" docs_count: int,\n"
" chunk_size: int = 500,\n"
" top_k: int = 5):\n"
" '''Estimate RAG pipeline cost.''' \n"
" # Embedding cost (one-time\n"
" # + new docs)\n"
" total_chunks = (\n"
" docs_count * 10) # approx\n"
" embedding_cost = (\n"
" total_chunks * chunk_size\n"
" / 1_000_000 * 0.02)\n"
" # $0.02/1M for embeddings\n"
" \n"
" # Query embedding cost\n"
" query_embed_cost = (\n"
" queries_per_day * 30\n"
" * 100 / 1_000_000 * 0.02)\n"
" \n"
" # LLM generation cost\n"
" # Input: query + top_k chunks\n"
" input_per_query = (\n"
" 200 + top_k * chunk_size)\n"
" output_per_query = 500\n"
" \n"
" llm_cost = (\n"
" estimate_monthly_cost(\n"
" model='llama-70b-self',\n"
" daily_users=queries_per_day,\n"
" sessions_per_user=1,\n"
" input_tokens_per_session=(\n"
" input_per_query),\n"
" output_tokens_per_session=(\n"
" output_per_query)))\n"
" \n"
" # Vector DB cost\n"
" vector_db = 45 # Qdrant/pgvector\n"
" \n"
" return {\n"
" 'embedding_one_time': round(\n"
" embedding_cost, 2),\n"
" 'query_embedding': round(\n"
" query_embed_cost, 2),\n"
" 'llm_generation': (\n"
" llm_cost['monthly_cost']),\n"
" 'vector_db': vector_db,\n"
" 'total_monthly': round(\n"
" query_embed_cost\n"
" + llm_cost['monthly_cost']\n"
" + vector_db, 2),\n"
" }"
)
def get_production_tips(self) -> dict:
"""Production tips."""
return {
"output_5x": "Output tokens cost 3-5x input at every vendor. Constrain output in prompt: 'respond in 200 tokens'.",
"agent_multiplier": "Agent multiplier is biggest cost lever. 1x simple RAG, 5-10x full agentic. Route sub-steps to economy tier.",
"cache_above_22": "Prompt caching saves above ~22% hit rate. Cache writes carry 1.25x premium. High-hit: 60-90% for stable system prompts.",
"batch_50": "Batch API: 50% off all token costs. Use for content generation, enrichment, classification, nightly reports.",
"reasoning_tokens": "Reasoning models: reasoning tokens can be 5-20x visible output. Budget for true all-in cost.",
"breakeven": "Self-hosting competitive at tens to hundreds of millions of tokens/month. Under-loaded GPU costs more per token than API.",
"four_layers": "Four cost layers: LLM API (80-95%), vector DB ($20-50), automation ($10+), monitoring. LLM dominates.",
"retry_buffer": "Budget 5-15% buffer for retries: rate limits, timeouts, model refusals. Measure real retry rate.",
"tier_routing": "Route 80% traffic to economy tier, 20% to workhorse/frontier. Biggest cost optimization lever.",
"benchmark_ratio": "Benchmark actual input/output ratio before projecting. Most production systems are input-heavy 10:1+.",
}
LLM Cost Estimation Checklist
- [ ] Count input tokens per request: system prompt + user message + retrieved context + tool definitions + conversation history
- [ ] Do not guess token counts from word count — run actual prompts through token estimator
- [ ] Typical input tokens: single chat 300-1500, RAG with retrievals 2K-8K, long-doc analysis 8K-100K
- [ ] Count output tokens per request — constrain in prompt: "respond in 200 tokens"
- [ ] Typical output tokens: classification 10-50, chat reply 150-600, code generation 500-2000, long-form 1000-4000
- [ ] Output tokens cost 3-5x more than input tokens at every major vendor — model them separately
- [ ] Use cost formula: (Users x DAU x Sessions x Input_Tokens x Input_Price) + (Output_Tokens x Output_Price) x Agent_Multiplier
- [ ] Apply retry rate buffer: 5-15% for rate limits, timeouts, model refusals
- [ ] Apply cache hit rate: 80-90% off cached input tokens, saves above ~22% hit rate
- [ ] Cache hit rate: 0% cold start, 40-60% warm cache for chatbots, 60-90% for stable system prompts
- [ ] Cache writes carry 1.25x premium at Anthropic — caching only saves above ~22% hit rate
- [ ] Apply batch API discount: 50% off all token costs for non-real-time tasks
- [ ] Batch API use cases: content generation, data enrichment, bulk classification, nightly reports
- [ ] Batch API: results returned within 24 hours, trade latency for cost
- [ ] Agent multiplier: simple RAG 1x, self-correcting RAG 3x, light agent 3-5x, full agentic 5-10x
- [ ] Agent multiplier is the biggest cost lever for agentic systems — budget upfront
- [ ] Route sub-steps to economy tier to reduce agent multiplier impact
- [ ] Model tier routing: economy (GPT-4o-mini $0.15/$0.60), workhorse (Claude Sonnet $3/$15), frontier (GPT-5 $5/$15)
- [ ] Flagship models cost 15-30x economy models — reserve for accuracy-critical tasks
- [ ] GPT-4o-mini: 80% of GPT-4o quality at 3% of cost — use for most tasks
- [ ] Four cost layers: LLM API (80-95% of total), vector database ($20-50/mo), automation platform, monitoring
- [ ] Vector database costs: Qdrant self-hosted $20-40/mo, Pinecone $50+/mo, Weaviate from $25/mo, pgvector free
- [ ] Embedding model cost: ~$0.02 per 1M tokens, one-time for document corpus + per-query
- [ ] Reasoning tokens: 5-20x visible output for reasoning models (o3, DeepSeek R1) — budget for true all-in cost
- [ ] Self-hosting break-even: competitive at tens to hundreds of millions of tokens per month
- [ ] Under 50M tokens/month: APIs almost always cheaper (including engineering time)
- [ ] 50-100M tokens/month: crossover zone, depends on GPU utilization
- [ ] Over 100M tokens/month: self-hosting usually wins
- [ ] Under-loaded GPU costs more per token than hosted API — factor in utilization
- [ ] Self-hosted: RTX 4090 $45/mo can serve 10M+ tokens (Llama 70B, Qwen 72B)
- [ ] Factor in MLOps engineering time for self-hosting — not just GPU + electricity
- [ ] Benchmark actual input/output ratio before projecting — most production systems are input-heavy 10:1+
- [ ] Days per month: 30 for 24/7 consumer apps, 22 for business-day-only internal tools
- [ ] Budget 5-15% buffer for retries — measure real retry rate in production
- [ ] Total cost of ownership: API cost + prompt engineering time + output review time + latency impact
- [ ] Semantic cache: cached responses under 100ms, reduces API cost 30-60%
- [ ] Read choose right LLM for model selection
- [ ] Read RAG vs fine-tuning for knowledge approach
- [ ] Read FastAPI tutorial for API setup
- [ ] Read monitoring for cost tracking
- [ ] Test: cost projection matches actual monthly spend within 15%
- [ ] Test: cache hit rate achieves expected savings
- [ ] Test: batch API reduces cost for non-real-time tasks
- [ ] Test: agent multiplier matches expected LLM calls per message
- [ ] Test: self-hosted break-even calculation matches actual GPU costs
- [ ] Test: tier routing reduces cost without quality degradation
- [ ] Document cost model, assumptions, tier routing, cache strategy, break-even analysis
FAQ
How do you calculate LLM API costs before building?
Multiply users by sessions by tokens by price, adjusting for retries, cache, and agent multiplier. SumatoSoft: "Monthly Cost = ((Users x DAU x Sessions/day x Avg_Input_Tokens x Input_Price) + (Users x DAU x Sessions/day x Avg_Output_Tokens x Output_Price) + (Retry_rate x Base_cost) - (Cache_hit_rate x Cacheable_cost)) x Agent_Multiplier. Six variables: active users, sessions, input and output tokens per session, retry rate, cache hit rate, agent multiplier." AICost.ai: "Cost = (input + output tokens) x model rate x requests x ~30 days. Input tokens: system prompt + user message + retrieved context + tool definitions + conversation history. Output tokens cost 3-5x more than input at every major vendor." Formula: (1) Count input tokens per request (prompt + context + history). (2) Count output tokens (constrain in prompt). (3) Multiply by daily requests x 30 days. (4) Apply cache hit rate discount (80-90% off cached). (5) Apply batch API discount (50% off). (6) Multiply by agent multiplier (1x simple RAG, 5-10x agentic).
What is the agent multiplier and how does it affect LLM costs?
Agent multiplier accounts for multiple LLM calls per user message in agentic systems. SumatoSoft: "Agent multiplier: simple RAG is 1x. Self-correcting RAG about 3x. Light agents with two or three tools 3-5x. Full agentic systems with planning and reasoning loops 5-10x. The agent multiplier dominates the bill for agentic systems." CreatorOpsMatrix: "LLM calls per message: 3 for typical agent. 100 users x 5 msg/day x 3 LLM calls/msg x 30 days = 45,000 calls/month. LLM cost dominates at $833/mo out of $868 total." Multipliers: (1) Simple RAG: 1x (one LLM call per query). (2) Self-correcting RAG: 3x (generate, evaluate, regenerate). (3) Light agent (2-3 tools): 3-5x. (4) Full agentic (planning, reasoning loops): 5-10x. (5) Budget for multiplier upfront — it is the biggest cost lever for agentic systems. (6) Route sub-steps to economy tier to reduce multiplier impact.
How much can you save with prompt caching and batch API?
Prompt caching saves 80-90% on cached input tokens. Batch API saves 50% on all token costs for non-real-time tasks. AICost.ai: "Cached reads cost 90% less. Cache writes carry 1.25x premium at Anthropic, so caching only saves money above ~22% hit rate. Batch API discounts are 50% off standard rates across providers." DevZone: "Cached input pricing applies when you reuse same context prefix across requests. Batch API pricing applies when you submit requests asynchronously in bulk (results within 24 hours). Both discounts typically offer 50-90% savings but serve different use cases." CreatorOpsMatrix: "If your agent LLM cost exceeds $200/month and any portion of calls are not time-sensitive, switching those to Batch API approximately halves that portion of your bill." Savings: (1) Prompt cache: 80-90% off cached input tokens. (2) Cache hit rate 40-60% for chatbots with stable system prompts. (3) Batch API: 50% off all token costs. (4) Batch for: content generation, enrichment, bulk classification, nightly reports. (5) Cache for: agent loops, RAG with stable retrieval, templated workflows. (6) Combine both for maximum savings.
When does self-hosting become cheaper than LLM APIs?
Self-hosting becomes competitive at tens to hundreds of millions of tokens per month. SumatoSoft: "Self-hosting turns competitive somewhere in the range of tens to hundreds of millions of tokens a month. Crossover depends heavily on GPU utilization — an under-loaded GPU can cost more per token than a hosted API. Below that, and for teams without MLOps capacity, hosted APIs usually win once you count engineering time." Gekro: "Break-even mode overlays cloud API costs against local hardware TCO and draws the crossover line. Hardware cost amortized linearly over chosen period. Total monthly cost = amortized hardware + electricity." CreatorOpsMatrix: "Qdrant self-hosted: $20-40/month. pgvector on Supabase: free for MVPs." Break-even: (1) Under 50M tokens/month: APIs almost always cheaper. (2) 50-100M tokens/month: crossover zone, depends on GPU utilization. (3) Over 100M tokens/month: self-hosting usually wins. (4) Factor in: GPU cost + electricity + MLOps engineering time. (5) Under-loaded GPU costs more per token than API. (6) RTX 4090: $45/mo can serve 10M+ tokens.
What are the hidden costs of running an AI application?
Beyond LLM API costs: vector database, embedding model, automation platform, monitoring, retry overhead, and engineering time. CreatorOpsMatrix: "Four cost layers: LLM API (biggest variable), vector database (RAG memory), automation platform (workflow orchestration), monitoring infrastructure. Qdrant self-hosted $20-40/mo, Pinecone $50+/mo, Weaviate from $25/mo. Total monthly AI agent cost: $868 — LLM $833, vector DB $25, automation $10." SumatoSoft: "Budget a 5-15% buffer for retries. Measure the real ratio. Output ~5x input, flagship ~15-30x economy, agent multiplier 1-10x." DevZone: "For o3, reasoning tokens can be 5-20x the length of visible output. A task that appears to cost $0.004 in output tokens may actually cost $0.04-0.08 when reasoning tokens are included." Hidden costs: (1) Vector database: $20-50+/month. (2) Embedding model: per-token or self-hosted. (3) Retry overhead: 5-15% buffer. (4) Reasoning tokens: 5-20x visible output for reasoning models. (5) Monitoring and observability infrastructure. (6) Prompt engineering and evaluation time. (7) MLOps engineering for self-hosted. (8) Data preparation and labeling.
Want a self-hosted AI company brain that does all of this out of the box?
Book a demo →