LLM Token Tracking and Cost: Monitor Spend Per Request with LiteLLM and Langfuse

By Fadi Updated Apr 17, 2026 12 min read LLM Gateways

TL;DR — LLM token tracking and cost: monitor spend per request. Langfuse: "Automatic cost calculation based on model and token counts. Integrations with LiteLLM, Langchain, LlamaIndex for automatic token usage collection." LiteLLM Docs: "Spend tracking with metadata tags. Per virtual key, per team, per user, per model. Admin dashboard at /ui. Budget enforcement blocks over-spend." TrueFoundry: "LiteLLM tracks spend per virtual API key or project, enforcing budgets in real time, shipping usage logs for downstream cost analysis." Maxim AI: "Top 5 tools: Bifrost, LiteLLM, Langfuse, Datadog, Langsmith. Ranked by cost attribution depth, enforcement, multi-provider, production readiness." CostGoat: "GPT-5: $10/$30 per 1M tokens. Every major provider offers per-token pricing." Learn more with LLM gateway, LiteLLM tutorial, rate limiting, and OpenRouter vs LiteLLM.

Maxim AI explains the importance: "This guide evaluates the five best LLM cost tracking tools in 2026 — Bifrost by Maxim AI, LiteLLM, Langfuse, Datadog, Langsmith — ranked by cost attribution depth, enforcement capabilities, multi-provider support, and production readiness."

LiteLLM Docs describes the feature: "Spend tracking with metadata tags for cost attribution. Each request includes metadata with tags for cost attribution. Track spend per virtual key, per team, per user, per model. Admin dashboard at /ui shows spend breakdown."

Token Tracking Architecture

flowchart TD subgraph App["Your Application"] Request["API Request
with metadata tags
user, team, model"] end subgraph Provider["LLM Provider"] API["Provider API
OpenAI, Anthropic, Google"] Usage["Response Usage Object
prompt_tokens
completion_tokens
total_tokens"] end subgraph Tracking["Token & Cost Tracking"] Extract["Extract Tokens
from response.usage
input + output tokens"] Pricing["Pricing Lookup
model → input price
model → output price
per 1M tokens"] Calculate["Cost Calculation
cost = (input * input_price
+ output * output_price) / 1M"] Store["Store in Database
PostgreSQL
with metadata tags
timestamp, user, team"] end subgraph LiteLLM["LiteLLM Proxy"] VirtualKeys["Virtual Keys
max_budget per key
budget_duration
rpm/tpm limits"] TeamBudgets["Team Budgets
per-team spending caps
real-time enforcement"] Dashboard["Admin Dashboard
/ui endpoint
spend per key/team/model
requests, tokens, costs"] Alerts["Budget Alerts
Slack webhook
email notifications
custom callbacks"] end subgraph Observability["Observability Tools"] Langfuse["Langfuse
open-source
automatic cost calc
integrations"] Datadog["Datadog
infrastructure metrics
LLM cost dashboards"] Langsmith["Langsmith
LangChain ecosystem
evaluation + cost"] end Request --> API API --> Usage Usage --> Extract Extract --> Pricing Pricing --> Calculate Calculate --> Store Store --> VirtualKeys Store --> TeamBudgets VirtualKeys --> Dashboard TeamBudgets --> Dashboard Dashboard --> Alerts Store --> Langfuse Store --> Datadog Store --> Langsmith

LLM API Pricing (per 1M tokens)

Model	Input Price	Output Price	Notes
OpenAI GPT-5	$10	$30	Flagship
OpenAI GPT-4o	$2.50	$10	Balanced
OpenAI GPT-4o mini	$0.15	$0.60	Budget
Anthropic Claude 4 Opus	$15	$75	Flagship
Anthropic Claude 3.5 Sonnet	$3	$15	Balanced
Google Gemini 2.0 Flash	$0.10	$0.40	Budget
DeepSeek R1	$0.55	$2.19	Open source

Cost Tracking Tools

Tool	Type	Key Feature	Best For
LiteLLM	Self-hosted proxy	Virtual keys + budget enforcement	Teams needing enforcement
Langfuse	Open-source observability	Automatic cost calculation	Teams needing analytics
Bifrost/Maxim AI	Managed platform	Cost attribution depth	Enterprise cost control
Datadog	Infrastructure monitoring	LLM cost dashboards	Teams already on Datadog
Langsmith	LangChain ecosystem	Evaluation + cost	LangChain users

Implementation

from dataclasses import dataclass
from typing import Optional
from enum import Enum

class Provider(Enum):
    OPENAI = "openai"
    ANTHROPIC = "anthropic"
    GOOGLE = "google"

@dataclass
class LLMTokenTrackingGuide:
    """LLM token tracking and cost implementation guide."""

    def get_pricing_table(self) -> dict:
        """Model pricing per 1M tokens."""
        return {
            "openai/gpt-5": {
                "input": 10.0, "output": 30.0},
            "openai/gpt-4o": {
                "input": 2.50, "output": 10.0},
            "openai/gpt-4o-mini": {
                "input": 0.15, "output": 0.60},
            "anthropic/claude-4-opus": {
                "input": 15.0, "output": 75.0},
            "anthropic/claude-3-5-sonnet": {
                "input": 3.0, "output": 15.0},
            "gemini/gemini-2.0-flash": {
                "input": 0.10, "output": 0.40},
            "deepseek/deepseek-r1": {
                "input": 0.55, "output": 2.19},
        }

    def get_cost_calculation(self) -> str:
        """Calculate cost per request."""
        return (
            "def calculate_cost(\n"
            "    model: str,\n"
            "    prompt_tokens: int,\n"
            "    completion_tokens: int\n"
            ") -> float:\n"
            "    '''Calculate cost in USD.'''\n"
            "    pricing = {\n"
            "        'gpt-4o': {\n"
            "            'input': 2.50,\n"
            "            'output': 10.0},\n"
            "        'claude-3-5-sonnet': {\n"
            "            'input': 3.0,\n"
            "            'output': 15.0},\n"
            "        'gemini-2.0-flash': {\n"
            "            'input': 0.10,\n"
            "            'output': 0.40},\n"
            "    }\n"
            "    prices = pricing.get(\n"
            "        model, {'input': 0, 'output': 0})\n"
            "    cost = (\n"
            "        prompt_tokens * prices['input']\n"
            "        + completion_tokens * \n"
            "            prices['output']\n"
            "    ) / 1_000_000\n"
            "    return cost\n"
            "\n"
            "# Usage\n"
            "response = client.chat\\\n"
            "    .completions.create(\n"
            "    model='gpt-4o',\n"
            "    messages=[...])\n"
            "cost = calculate_cost(\n"
            "    'gpt-4o',\n"
            "    response.usage.prompt_tokens,\n"
            "    response.usage\n"
            "        .completion_tokens)\n"
            "print(f'Cost: ${cost:.6f}')"
        )

    def get_litellm_tracking(self) -> str:
        """LiteLLM spend tracking."""
        return (
            "# LiteLLM proxy tracks spend\n"
            "# automatically per request\n"
            "\n"
            "# Request with metadata tags\n"
            "import requests\n"
            "\n"
            "response = requests.post(\n"
            "    'http://localhost:4000'\n"
            "    '/chat/completions',\n"
            "    headers={\n"
            "        'Authorization':\n"
            "            'Bearer sk-1234'},\n"
            "    json={\n"
            "        'model': 'gpt-4o',\n"
            "        'messages': [\n"
            "            {'role': 'user',\n"
            "             'content': 'Hello!'}\n"
            "        ],\n"
            "        'metadata': {\n"
            "            'tags': [\n"
            "                'app-prod',\n"
            "                'team-frontend'\n"
            "            ],\n"
            "            'user_id': 'user-123',\n"
            "            'team_id': 'team-abc',\n"
            "        }\n"
            "    }\n"
            ")\n"
            "# LiteLLM logs:\n"
            "# - prompt_tokens\n"
            "# - completion_tokens\n"
            "# - cost (auto-calculated)\n"
            "# - metadata tags\n"
            "# - timestamp\n"
            "# - virtual key used\n"
            "# - model called\n"
            "\n"
            "# View spend in dashboard\n"
            "# http://localhost:4000/ui"
        )

    def get_virtual_keys_budget(self) -> str:
        """Virtual keys with budget enforcement."""
        return (
            "import requests\n"
            "\n"
            "# Create virtual key with budget\n"
            "response = requests.post(\n"
            "    'http://localhost:4000'\n"
            "    '/key/generate',\n"
            "    headers={\n"
            "        'Authorization':\n"
            "            'Bearer sk-1234'},\n"
            "    json={\n"
            "        'key_alias':\n"
            "            'prod-api-key',\n"
            "        'max_budget': 500.0,\n"
            "        'budget_duration':\n"
            "            'monthly',\n"
            "        'rpm_limit': 100,\n"
            "        'tpm_limit': 100000,\n"
            "        'models': [\n"
            "            'gpt-4o',\n"
            "            'claude-3-5-sonnet'\n"
            "        ],\n"
            "        'team_id': 'team-abc',\n"
            "    }\n"
            ")\n"
            "key = response.json()['key']\n"
            "\n"
            "# Budget enforcement:\n"
            "# - Requests blocked when\n"
            "#   max_budget exceeded\n"
            "# - Rate limits enforced\n"
            "#   (rpm, tpm)\n"
            "# - Model access restricted\n"
            "# - Team budget tracked\n"
            "\n"
            "# Check spend\n"
            "response = requests.get(\n"
            "    'http://localhost:4000'\n"
            "    '/key/info',\n"
            "    headers={\n"
            "        'Authorization':\n"
            "            f'Bearer {key}'})\n"
            "print(response.json())\n"
            "# Shows: spend, budget,\n"
            "# remaining, usage stats"
        )

    def get_langfuse_integration(self) -> str:
        """Langfuse cost tracking integration."""
        return (
            "from langfuse.openai import (\n"
            "    openai)\n"
            "\n"
            "# Langfuse OpenAI wrapper\n"
            "# automatically tracks:\n"
            "# - token usage\n"
            "# - cost calculation\n"
            "# - model used\n"
            "# - metadata\n"
            "\n"
            "response = openai.chat\\\n"
            "    .completions.create(\n"
            "    model='gpt-4o',\n"
            "    messages=[\n"
            "        {'role': 'user',\n"
            "         'content': 'Hello!'}\n"
            "    ],\n"
            "    metadata={\n"
            "        'user_id': 'user-123',\n"
            "        'session_id': 'sess-456',\n"
            "        'tags': ['prod', 'chat']\n"
            "    }\n"
            ")\n"
            "# Langfuse dashboard shows:\n"
            "# - cost per request\n"
            "# - cost per user\n"
            "# - cost per session\n"
            "# - cost per model\n"
            "# - token usage trends\n"
            "# - cost over time"
        )

    def get_budget_alerts(self) -> dict:
        """Budget alert configuration."""
        return {
            "slack_alerts": (
                "# config.yaml\n"
                "general_settings:\n"
                "  alerting: ['slack']\n"
                "  alerting_threshold: 100\n"
                "  # Alert when spend\n"
                "  # exceeds $100"
            ),
            "alert_types": [
                "Budget exceeded — virtual key blocked",
                "Spend threshold — warning at X% of budget",
                "Rate limit hit — RPM or TPM exceeded",
                "Model error — provider failure",
                "Daily spend report — summary",
            ],
            "alert_channels": [
                "Slack webhook — real-time notifications",
                "Email — daily/weekly summaries",
                "Custom callbacks — programmatic alerts",
                "Admin dashboard — visual indicators",
            ],
        }

    def get_database_schema(self) -> str:
        """PostgreSQL schema for spend tracking."""
        return (
            "-- LiteLLM spend tracking schema\n"
            "CREATE TABLE spend_logs (\n"
            "  id SERIAL PRIMARY KEY,\n"
            "  timestamp TIMESTAMP DEFAULT\n"
            "    NOW(),\n"
            "  model VARCHAR(100),\n"
            "  api_key VARCHAR(100),\n"
            "  prompt_tokens INT,\n"
            "  completion_tokens INT,\n"
            "  total_tokens INT,\n"
            "  spend DECIMAL(10, 6),\n"
            "  metadata JSONB,\n"
            "  team_id VARCHAR(100),\n"
            "  user_id VARCHAR(100),\n"
            "  tags TEXT[]\n"
            ");\n"
            "\n"
            "-- Aggregate views\n"
            "CREATE VIEW daily_spend AS\n"
            "SELECT DATE(timestamp) as date,\n"
            "  model,\n"
            "  SUM(spend) as total_spend,\n"
            "  SUM(total_tokens) as tokens,\n"
            "  COUNT(*) as requests\n"
            "FROM spend_logs\n"
            "GROUP BY DATE(timestamp), model;\n"
            "\n"
            "-- Per-team spend\n"
            "CREATE VIEW team_spend AS\n"
            "SELECT team_id,\n"
            "  SUM(spend) as total_spend,\n"
            "  SUM(total_tokens) as tokens,\n"
            "  COUNT(*) as requests\n"
            "FROM spend_logs\n"
            "GROUP BY team_id;"
        )

LLM Token Tracking and Cost Checklist

[ ] Extract token usage from API response: prompt_tokens, completion_tokens, total_tokens
[ ] Look up model pricing: input price per 1M, output price per 1M
[ ] Calculate cost: (input_tokens * input_price + output_tokens * output_price) / 1M
[ ] Store spend in database with metadata: user, team, model, timestamp, tags
[ ] LiteLLM proxy tracks spend automatically per request in PostgreSQL
[ ] LiteLLM: metadata tags for cost attribution (app, team, user, environment)
[ ] LiteLLM: admin dashboard at /ui shows spend breakdown
[ ] LiteLLM: spend per virtual key, per team, per user, per model
[ ] Virtual keys: max_budget parameter for dollar limit per key
[ ] Virtual keys: budget_duration (daily, weekly, monthly)
[ ] Virtual keys: rpm_limit and tpm_limit for rate limits
[ ] Virtual keys: models list to restrict model access
[ ] Virtual keys: team_id for per-team budgets
[ ] Budget enforcement: requests blocked when max_budget exceeded
[ ] Budget alerts: Slack webhook for budget alerts and errors
[ ] Budget alerts: alerting_threshold for spend warnings
[ ] Budget alerts: daily spend reports, threshold warnings
[ ] Alert types: budget exceeded, spend threshold, rate limit, model error
[ ] Alert channels: Slack, email, custom callbacks, admin dashboard
[ ] Langfuse: automatic cost calculation based on model and token counts
[ ] Langfuse: OpenAI wrapper collects token usage automatically
[ ] Langfuse: integrations with LiteLLM, Langchain, LlamaIndex
[ ] Langfuse: open-source, self-hosted observability
[ ] Langfuse: cost per request, per user, per session, per model
[ ] Top 5 cost tracking tools: Bifrost/Maxim AI, LiteLLM, Langfuse, Datadog, Langsmith
[ ] Ranked by: cost attribution depth, enforcement capabilities, multi-provider, production readiness
[ ] LLM pricing: GPT-5 $10/$30, GPT-4o $2.50/$10, Claude 4 Opus $15/$75 per 1M tokens
[ ] LLM pricing: Claude 3.5 Sonnet $3/$15, Gemini 2.0 Flash $0.10/$0.40 per 1M tokens
[ ] LLM pricing: DeepSeek R1 $0.55/$2.19 per 1M tokens
[ ] Flagship models 10-100x more expensive than flash/lite variants
[ ] Track input and output tokens separately for accurate cost calculation
[ ] PostgreSQL schema: spend_logs table with model, tokens, spend, metadata, team_id
[ ] PostgreSQL: aggregate views for daily spend, team spend, model spend
[ ] Cost attribution: per-key, per-team, per-user, per-model, per-tag
[ ] Spend tracking: real-time enforcement, not just post-hoc analysis
[ ] LiteLLM: DATABASE_URL env var for PostgreSQL connection
[ ] LiteLLM: /key/info endpoint to check remaining budget
[ ] Datadog: infrastructure monitoring with LLM cost dashboards
[ ] Langsmith: LangChain ecosystem with evaluation + cost tracking
[ ] Cost optimization: route to cheaper models for simple tasks
[ ] Cost optimization: cache responses to avoid redundant API calls
[ ] Cost optimization: use flash/lite variants for non-critical tasks
[ ] Read LLM gateway guide for gateway concepts
[ ] Read LiteLLM tutorial for proxy setup
[ ] Read rate limiting for RPM/TPM limits
[ ] Read OpenRouter vs LiteLLM for comparison
[ ] Test: cost calculation matches provider billing
[ ] Test: virtual key budget enforcement blocks over-spend
[ ] Test: metadata tags correctly attributed in dashboard
[ ] Test: Slack alert fires when threshold exceeded
[ ] Test: Langfuse integration captures token usage automatically
[ ] Test: PostgreSQL aggregate views show correct spend totals
[ ] Document pricing table, budget config, alert thresholds, and cost attribution strategy

FAQ

How do you track LLM token usage and cost per request?

Extract token counts from API response usage object and multiply by model pricing. Langfuse: "Provide token usage when ingesting generations. When utilizing Langfuse OpenAI wrapper or integrations such as LiteLLM, Langchain, LlamaIndex, token usage is collected and provided automatically. Automatic cost calculation based on model and token counts." LiteLLM Docs: "Spend tracking with metadata tags for cost attribution. Each request includes usage object with prompt_tokens, completion_tokens, total_tokens. LiteLLM calculates cost automatically based on model pricing." Per-request tracking: (1) Extract usage from response: prompt_tokens, completion_tokens. (2) Look up model pricing (input per 1M, output per 1M). (3) Calculate: cost = (input_tokens * input_price + output_tokens * output_price) / 1M. (4) Store in database with metadata (user, team, model, timestamp). (5) Aggregate for dashboards and alerts.

How does LiteLLM spend tracking work?

LiteLLM proxy tracks spend per request in PostgreSQL with virtual keys, per-team budgets, and admin dashboard. LiteLLM Docs: "Spend tracking with metadata tags. Each request includes metadata with tags for cost attribution. Track spend per virtual key, per team, per user, per model. Admin dashboard at /ui shows spend breakdown." TrueFoundry: "LiteLLM tracks spend per virtual API key or project, enforcing budgets in real time and shipping usage logs for downstream cost analysis. You pay each underlying provider directly." Maxim AI: "LiteLLM ranked #2 for cost tracking. Automatic cost calculation based on model and token counts. Cost attribution depth and enforcement capabilities." Setup: (1) DATABASE_URL for PostgreSQL. (2) Virtual keys with max_budget. (3) Per-team budgets in config.yaml. (4) Spend logged per request. (5) Admin dashboard at /ui. (6) Budget alerts via Slack webhook.

What are current LLM API pricing rates?

LLM APIs charge per-token with different input and output rates. CostGoat: "Every major AI provider — OpenAI, Anthropic, Google, DeepSeek, Mistral, xAI — offers API access with per-token pricing. OpenAI GPT-5 costs roughly $10/$30 per 1M tokens (input/output)." Pricing examples (per 1M tokens): OpenAI GPT-5: $10 input / $30 output. OpenAI GPT-4o: $2.50 input / $10 output. Anthropic Claude 4 Opus: $15 input / $75 output. Anthropic Claude 3.5 Sonnet: $3 input / $15 output. Google Gemini 2.0 Flash: $0.10 input / $0.40 output. DeepSeek R1: $0.55 input / $2.19 output. Costs vary by model capability — flagship models 10-100x more expensive than flash/lite variants. Track both input and output tokens separately for accurate cost calculation.

How do you set up budget alerts and enforcement for LLM spend?

Use LiteLLM virtual keys with max_budget, per-team budgets, and Slack alerts. LiteLLM Docs: "Virtual keys with max_budget parameter. Budget enforcement: requests blocked when budget exceeded. Per-team budgets with team_id. Alerting via Slack webhook for budget alerts and errors." Maxim AI: "LiteLLM enforcement capabilities — budget enforcement blocks over-spend in real time. Cost attribution depth — per-key, per-team, per-user, per-model." Budget setup: (1) Create virtual key with max_budget (dollar limit). (2) Set budget_duration (daily, weekly, monthly). (3) Set rpm_limit and tpm_limit for rate limits. (4) Assign to team with team budget. (5) Configure Slack webhook for alerts. (6) Requests blocked when budget exceeded. (7) Admin dashboard shows remaining budget. (8) Alerting: Slack webhook, email, custom callbacks.

What are the best LLM cost tracking tools?

Top tools include LiteLLM, Langfuse, Bifrost by Maxim AI, Datadog, and Langsmith. Maxim AI: "Five best LLM cost tracking tools in 2026: 1) Bifrost by Maxim AI, 2) LiteLLM, 3) Langfuse, 4) Datadog, 5) Langsmith. Ranked by cost attribution depth, enforcement capabilities, multi-provider support, and production readiness." Langfuse: "Automatic cost calculation based on model and token counts. Integrations with LiteLLM, Langchain, LlamaIndex for automatic token usage collection. Open-source, self-hosted." Tools: LiteLLM (proxy with spend tracking, virtual keys, budget enforcement, admin dashboard), Langfuse (open-source observability with cost tracking, integrations), Bifrost/Maxim AI (cost attribution depth, enforcement), Datadog (infrastructure monitoring with LLM metrics), Langsmith (LangChain ecosystem, evaluation + cost). Choose based on: self-hosted vs managed, enforcement needs, multi-provider support, integration ecosystem.

Want a self-hosted AI company brain that does all of this out of the box?
Book a demo →