AI Budget Caps: Enforcing Spending Limits on LLM Costs in Production

TL;DR — AI budget caps prevent LLM cost overruns by enforcing spending limits before requests execute. Portkey: "Hard caps block requests, soft caps trigger alerts. Budgets can be daily, weekly, or monthly." TrueFoundry: "Spending limits configured per team, service, and endpoint, enforced before execution. Overruns get prevented rather than flagged after the invoice arrives." Waxell: "AI agent token budget enforcement terminates or pauses an agent when it reaches a defined threshold. Unlike alerts — which notify after spending — enforcement stops the agent: no further LLM calls until a human or policy resumes." Cloudflare: "Set per-user budgets — $500/month for ICs, $2,000 for senior engineers. When a user hits their limit, requests can be downgraded to a cheaper model or blocked." Maxim: "Visibility tools tell you spend climbed; enforcement tools prevent it from climbing past the cap." Budget levels: per-user, per-team, per-tenant, per-application, global. Enforcement: hard cap (block or downgrade), soft cap (alert). Integrate with usage tracking, LLM cost optimization, per-tenant quotas, and enterprise TCO.

Maxim frames the problem: "Visibility tools tell you that spend climbed; enforcement tools prevent it from climbing past the cap in the first place. As enterprise AI spend continues to scale — Menlo's mid-year 2025 update tracked LLM API spend doubling in six months — the cost of relying on retroactive observability alone has become unsustainable."

Waxell describes the gap: "The $400M AI FinOps gap is the difference between cost visibility (knowing what you spent) and cost enforcement (preventing overspend). Enterprises spend on visibility (dashboards, analytics) but not on enforcement (hard caps, agent termination, model downgrade). Closing the gap requires moving from monitoring to prevention."

AI Budget Caps Architecture

Budget Cap Types and Actions

Budget Levels and Typical Limits

Implementation

import time
import json
from datetime import datetime, timezone
from dataclasses import dataclass
from typing import Optional, Literal

@dataclass
class BudgetCapEnforcer:
    """AI budget cap enforcement: hard caps, soft caps, agent budgets."""

    # Budget configuration
    soft_cap_thresholds = [0.80, 0.90]  # alert at 80% and 90%
    hard_cap_threshold = 1.0  # block or downgrade at 100%

    # Default budgets (can be overridden per tenant)
    default_budgets = {
        "per_user": {"daily": 50.0, "monthly": 500.0},
        "per_team": {"daily": 500.0, "monthly": 10000.0},
        "per_tenant": {"daily": 1000.0, "monthly": 25000.0},
        "per_application": {"daily": 200.0, "monthly": 5000.0},
        "global": {"daily": 5000.0, "monthly": 100000.0},
    }

    # Agent session budgets
    default_agent_token_budget = 100000  # tokens per session

    # Enforcement mode
    enforcement_mode: Literal["monitor", "enforce"] = "enforce"
    hard_cap_action: Literal["block", "downgrade"] = "downgrade"

    def __init__(self, redis, audit, alert_service, llm_router):
        self.redis = redis
        self.audit = audit
        self.alerts = alert_service
        self.router = llm_router

    async def check_budget(self, tenant_id: str, user_id: str,
                           team_id: str, application: str,
                           estimated_cost: float = 0.01) -> dict:
        """Check all budget levels before allowing an LLM call."""

        checks = []

        # Check each budget level
        for level in ["per_user", "per_team", "per_tenant",
                      "per_application", "global"]:
            entity_id = self._get_entity_id(
                level, tenant_id, user_id, team_id, application)
            budget = await self._get_budget(level, tenant_id, entity_id)

            check = await self._check_level(
                level, tenant_id, entity_id, budget, estimated_cost)
            checks.append(check)

        # Determine overall action
        any_hard_cap = any(c["action"] == "block" for c in checks)
        any_downgrade = any(c["action"] == "downgrade" for c in checks)
        any_alert = any(c["action"] == "alert" for c in checks)

        if any_hard_cap:
            action = "block"
        elif any_downgrade:
            action = "downgrade"
        elif any_alert:
            action = "alert"
        else:
            action = "allow"

        return {
            "action": action,
            "checks": checks,
            "tenant_id": tenant_id,
            "user_id": user_id,
        }

    async def _check_level(self, level: str, tenant_id: str,
                           entity_id: str, budget: dict,
                           estimated_cost: float) -> dict:
        """Check a single budget level."""
        date_key = datetime.now(timezone.utc).strftime("%Y-%m-%d")
        month_key = datetime.now(timezone.utc).strftime("%Y-%m")

        daily_spent = float(await self.redis.get(
            f"budget:{level}:{tenant_id}:{entity_id}:daily:{date_key}") or 0)
        monthly_spent = float(await self.redis.get(
            f"budget:{level}:{tenant_id}:{entity_id}:monthly:{month_key}") or 0)

        daily_pct = daily_spent / budget["daily"]
        monthly_pct = monthly_spent / budget["monthly"]
        max_pct = max(daily_pct, monthly_pct)

        result = {
            "level": level,
            "entity_id": entity_id,
            "daily_spent": round(daily_spent, 2),
            "daily_limit": budget["daily"],
            "monthly_spent": round(monthly_spent, 2),
            "monthly_limit": budget["monthly"],
            "utilization_pct": round(max_pct * 100, 1),
            "action": "allow",
        }

        if self.enforcement_mode == "monitor":
            # Only alert, never block
            for threshold in self.soft_cap_thresholds:
                if max_pct >= threshold:
                    result["action"] = "alert"
                    await self._send_alert(
                        tenant_id, level, entity_id, result, threshold)
                    break
            return result

        # Enforcement mode
        if max_pct >= self.hard_cap_threshold:
            if self.hard_cap_action == "block":
                result["action"] = "block"
            else:
                result["action"] = "downgrade"
            await self.audit.log(
                tenant_id=tenant_id,
                action=f"budget_hard_cap_{self.hard_cap_action}",
                entity_type="budget_cap",
                after=result,
            )
        else:
            for threshold in self.soft_cap_thresholds:
                if max_pct >= threshold:
                    result["action"] = "alert"
                    await self._send_alert(
                        tenant_id, level, entity_id, result, threshold)
                    break

        return result

    async def record_spend(self, tenant_id: str, user_id: str,
                           team_id: str, application: str,
                           cost: float):
        """Record actual spend after LLM call completes."""
        date_key = datetime.now(timezone.utc).strftime("%Y-%m-%d")
        month_key = datetime.now(timezone.utc).strftime("%Y-%m")

        for level in ["per_user", "per_team", "per_tenant",
                      "per_application", "global"]:
            entity_id = self._get_entity_id(
                level, tenant_id, user_id, team_id, application)

            await self.redis.incrbyfloat(
                f"budget:{level}:{tenant_id}:{entity_id}:daily:{date_key}",
                cost)
            await self.redis.expire(
                f"budget:{level}:{tenant_id}:{entity_id}:daily:{date_key}",
                172800)

            await self.redis.incrbyfloat(
                f"budget:{level}:{tenant_id}:{entity_id}:monthly:{month_key}",
                cost)
            await self.redis.expire(
                f"budget:{level}:{tenant_id}:{entity_id}:monthly:{month_key}",
                2764800)

    async def check_agent_budget(self, session_id: str,
                                 estimated_tokens: int) -> dict:
        """Check if agent session is within token budget."""
        used = int(await self.redis.get(
            f"agent_budget:{session_id}:used") or 0)
        budget = int(await self.redis.get(
            f"agent_budget:{session_id}:limit") or
            self.default_agent_token_budget)

        remaining = budget - used
        will_exceed = (used + estimated_tokens) > budget
        near_limit = remaining < (budget * 0.10)  # under 10% remaining

        if will_exceed:
            action = "terminate"
        elif near_limit:
            action = "pause"
        else:
            action = "allow"

        return {
            "session_id": session_id,
            "token_budget": budget,
            "tokens_used": used,
            "tokens_remaining": remaining,
            "estimated_next_call": estimated_tokens,
            "action": action,
        }

    async def record_agent_usage(self, session_id: str,
                                 tokens_used: int):
        """Record token usage for agent session."""
        await self.redis.incrby(
            f"agent_budget:{session_id}:used", tokens_used)

    async def set_agent_budget(self, session_id: str,
                               token_limit: int):
        """Set token budget for an agent session."""
        await self.redis.set(
            f"agent_budget:{session_id}:limit", token_limit)
        await self.redis.set(
            f"agent_budget:{session_id}:used", 0)

    async def _get_budget(self, level: str, tenant_id: str,
                          entity_id: str) -> dict:
        """Get budget limits for a level (with tenant overrides)."""
        # Check for tenant-specific overrides
        override = await self.redis.get(
            f"budget_config:{tenant_id}:{level}:{entity_id}")
        if override:
            return json.loads(override)
        return self.default_budgets[level]

    async def _send_alert(self, tenant_id: str, level: str,
                          entity_id: str, result: dict,
                          threshold: float):
        """Send budget alert via configured channels."""
        await self.alerts.send(
            tenant_id=tenant_id,
            channel="slack",
            message=(
                f"Budget alert: {level} {entity_id} at "
                f"{result['utilization_pct']}% utilization "
                f"(threshold: {threshold*100}%)"
            ),
            metadata=result,
        )

    def _get_entity_id(self, level: str, tenant_id: str,
                       user_id: str, team_id: str,
                       application: str) -> str:
        """Get entity ID for a budget level."""
        if level == "per_user":
            return user_id
        elif level == "per_team":
            return team_id
        elif level == "per_tenant":
            return tenant_id
        elif level == "per_application":
            return application
        elif level == "global":
            return "global"
        return "unknown"

AI Budget Caps Checklist

• [ ] Set per-user budget limits ($500/month ICs, $2K/month senior engineers)
• [ ] Set per-team budget limits (department-level caps)
• [ ] Set per-tenant budget limits (multi-tenant customer caps)
• [ ] Set per-application budget limits (feature-level caps)
• [ ] Set global organization-wide budget ceiling
• [ ] Set daily and monthly budget periods
• [ ] Configure soft cap thresholds (80% alert, 90% urgent alert)
• [ ] Configure hard cap action: block (return 429) or downgrade (cheaper model)
• [ ] Start with monitoring mode (high limit) before enforcing
• [ ] Switch to enforcement mode after understanding usage patterns
• [ ] Track cumulative spend in Redis (real-time counters)
• [ ] Check budget before each LLM call, not after
• [ ] Record actual spend after LLM call completes
• [ ] Reset daily counters at midnight, monthly counters on 1st
• [ ] Send alerts via Slack, email, and webhooks
• [ ] Alert at 80% (early warning), 90% (urgent), 100% (cap hit)
• [ ] Implement model downgrade when hard cap hit (frontier to cheap)
• [ ] Implement hard block when hard cap hit (return 429 with reason)
• [ ] Log all budget enforcement actions in audit log
• [ ] Track budget rejection rate as a metric
• [ ] Set agent session token budgets (50K-500K tokens per session)
• [ ] Track cumulative token usage per agent session
• [ ] Estimate next call cost before executing
• [ ] Terminate agent session when token budget exceeded
• [ ] Pause agent and request human approval when near limit
• [ ] Enforce agent budgets before LLM call, not after
• [ ] Support tenant-specific budget overrides
• [ ] Allow admins to adjust budgets dynamically
• [ ] Allow temporary budget increases with approval workflow
• [ ] Track which users/teams consistently hit caps (training opportunity)
• [ ] Provide self-service budget view for users (see remaining budget)
• [ ] Provide team-level budget dashboard for managers
• [ ] Provide tenant-level budget dashboard for customers
• [ ] Implement budget forecasting based on usage trends
• [ ] Alert on unusual spending patterns (anomaly detection)
• [ ] Integrate with AI usage tracking for data
• [ ] Apply LLM cost optimization to reduce spend
• [ ] Track per-tenant cost with caps
• [ ] Calculate enterprise AI TCO with caps
• [ ] Avoid AI cost traps with caps
• [ ] Consider BYOK to shift cost to customers
• [ ] Implement LLM usage metering with caps
• [ ] Log all cap events in audit logs
• [ ] Apply RBAC to budget management
• [ ] Apply multi-tenant budget isolation
• [ ] Set up platform monitoring for budgets
• [ ] Test: hard cap blocks requests correctly
• [ ] Test: soft cap sends alerts at correct thresholds
• [ ] Test: model downgrade routes to cheaper model
• [ ] Test: agent session terminates at token budget
• [ ] Test: budget counters reset correctly
• [ ] Document budget cap configuration and enforcement policy

FAQ

What are AI budget caps?

AI budget caps are spending limits that prevent LLM costs from exceeding defined thresholds. Portkey: "Budgets can be daily, weekly, or monthly, and depending on your risk tolerance, you can set hard caps (where requests are blocked) or soft caps (where alerts are triggered but usage can continue)." TrueFoundry: "Per-team and per-application token budgets with hard limits: spending limits get configured per team, service, and endpoint, then enforced before execution. Overruns get prevented rather than flagged after the invoice arrives." Maxim: "Visibility tools tell you that spend climbed; enforcement tools prevent it from climbing past the cap in the first place." Budget levels: (1) Per-user — individual spending limits. (2) Per-team — department budgets. (3) Per-tenant — multi-tenant customer limits. (4) Per-application — feature-level caps. (5) Global — organization-wide ceiling.

What is the difference between hard caps and soft caps?

Hard caps block requests when the budget is exceeded, while soft caps trigger alerts but allow usage to continue. Portkey: "Hard caps: where requests are blocked. Soft caps: where alerts are triggered but usage can continue." Waxell: "When the session cumulative token consumption crosses the threshold, the session is terminated before the next LLM call initiates — not after. The threshold is defined at the governance layer and enforced by the runtime, independent of the agent reasoning. This is distinct from account-level API rate limits, which throttle throughput but do not stop runaway spend." Cloudflare: "When a user hits their limit, requests can be downgraded to a cheaper model or blocked." Implementation: (1) Hard cap at 100% — block all requests or downgrade to cheaper model. (2) Soft cap at 80% — alert admin and user. (3) Soft cap at 90% — alert with urgency, suggest optimization. (4) Hard cap at 100% — block or downgrade. Start with monitoring mode (high limit) before enforcing.

How do you enforce token budgets on AI agents?

Enforce token budgets on AI agents with runtime controls that terminate or pause agents when they reach defined thresholds. Waxell: "AI agent token budget enforcement is the set of runtime controls that terminate or pause an agent when it reaches a defined token or cost threshold. Unlike budget alerts — which notify after spending occurs — enforcement stops the agent: no further LLM calls are placed until a human or policy resumes." Implementation: (1) Set token budget per agent session (e.g., 50K tokens). (2) Track cumulative token consumption per session in real-time. (3) Before each LLM call, check if remaining budget covers estimated cost. (4) If exceeded: terminate session, pause agent, or request human approval. (5) Log all enforcement actions for audit. Waxell: "The threshold is defined at the governance layer and enforced by the runtime, independent of the agent reasoning. This is distinct from account-level API rate limits, which throttle throughput but do not stop runaway spend." Key: enforcement must happen before the LLM call, not after.

How do you implement per-user and per-team budget caps?

Implement per-user and per-team budget caps by tracking cumulative spend in real-time and checking limits before each LLM call. Cloudflare: "AI Gateway calculates cost per request based on model pricing, and tracks cumulative spend against your limit in real time. Set per-user budgets — $500/month for individual contributors and $2,000 for senior engineers. When a user hits their limit, requests can be downgraded to a cheaper model or blocked." TrueFoundry: "Spending limits get configured per team, service, and endpoint, then enforced before execution." Implementation: (1) Store cumulative spend per user/team in Redis (daily and monthly counters). (2) Before each LLM call, check if user/team is within budget. (3) If exceeded: block (return 429 error), downgrade (route to cheaper model), or alert (notify admin). (4) Reset counters daily/monthly. (5) Track budget rejection rate as a metric. Behest: "Daily and monthly caps at the global, tenant, and project level. Stop overruns before the invoice lands. Token budgets enforced inline."

What is the $400M AI FinOps gap?

The $400M AI FinOps gap is the difference between cost visibility (knowing what you spent) and cost enforcement (preventing overspend). Waxell: "Visibility tools tell you that spend climbed; enforcement tools prevent it from climbing past the cap in the first place. The cost of relying on retroactive observability alone has become unsustainable." Maxim: "As enterprise AI spend continues to scale — Menlo mid-year 2025 update tracked LLM API spend doubling in six months — the cost of relying on retroactive observability alone has become unsustainable." InformationWeek: "AI end-users within an organization often have a limited understanding of how LLMs are priced or how user activities impact total spending. Lack of FinOps tools for LLMs — while FinOps offers mature solutions for cloud spending, equivalent tools for LLM cost management are still emerging." The gap: enterprises spend on visibility (dashboards, analytics) but not on enforcement (hard caps, agent termination, model downgrade). Closing the gap requires moving from monitoring to prevention.

Want a self-hosted AI company brain that does all of this out of the box?
Book a demo →