AWS Bedrock with LiteLLM: Proxy Gateway for Claude, Llama, and Titan Models

By Fadi Updated Dec 31, 2025 11 min read LLM Gateways

TL;DR — AWS Bedrock with LiteLLM: proxy gateway for Claude, Llama, Titan. LiteLLM Docs: "boto3 required. Routes: bedrock/, bedrock/converse/, bedrock/invoke/. Endpoints: /chat/completions, /embeddings, /images, /rerank. Auth: aws_access_key_id, aws_secret_access_key, aws_region_name, STS AssumeRole, IAM Roles Anywhere." AWS Samples: "Routing: load balancing, fallback, rate-limit-aware, latency-based. Fallbacks: claude-3-sonnet → claude-3-haiku. tpm/rpm per deployment. enable_pre_call_checks." LiteLLM Docs: "Multi-region load balancing: same model_name, different aws_region_name. Boto3 SDK: endpoint_url=localhost:4000/bedrock, AWS_BEARER_TOKEN_BEDROCK=sk-1234." Learn more with LLM gateway, LiteLLM tutorial, fallbacks, and OpenAI compatible API.

LiteLLM Docs describes the integration: "Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs). Provider routes on LiteLLM: bedrock/, bedrock/converse/, bedrock/invoke/, bedrock/converse_like/, bedrock/llama/, bedrock/deepseek_r1/, bedrock/qwen3/, bedrock/qwen2/, bedrock/openai/, bedrock/moonshot."

AWS Samples explains the routing: "LiteLLM's routing feature allows for dynamic control over how requests are directed to LLMs deployed across multiple backends. The routing configuration is critical for optimizing load distribution, cost management, fallback strategies, and latency."

Bedrock + LiteLLM Architecture

flowchart TD subgraph App["Your Application"] SDK["OpenAI SDK or boto3
base_url: localhost:4000
api_key: LiteLLM key"] end subgraph LiteLLM["LiteLLM Proxy"] Config["config.yaml
bedrock/ prefix models
aws_region_name per deployment"] Auth["Boto3 Authentication
AWS credentials
STS AssumeRole
IAM Roles Anywhere"] Router["Router
load balancing across regions
latency-based routing
rate-limit-aware (tpm/rpm)"] Fallback["Fallbacks
claude-3-sonnet → claude-3-haiku
model-level failover"] Tracking["Spend Tracking
PostgreSQL
virtual keys with budgets
admin dashboard"] end subgraph Bedrock["AWS Bedrock"] Converse["Converse API
bedrock/converse/ prefix
unified conversation interface"] Invoke["Invoke API
bedrock/invoke/ prefix
raw model invocation"] Regions["Multi-Region
us-west-2
us-east-1
us-east-2"] end subgraph Models["Bedrock Foundation Models"] Claude["Anthropic Claude
claude-3.5-sonnet
claude-3-haiku
claude-instant"] Llama["Meta Llama
llama3-70b-instruct
llama3-8b-instruct"] Titan["Amazon Titan
titan-text-express
titan-embeddings"] Cohere["Cohere
command-r-v1
command-text-v14"] Others["Others
DeepSeek R1
Qwen3, Moonshot"] end SDK --> Config Config --> Auth Auth --> Router Router --> Fallback Fallback --> Tracking Router --> Converse Router --> Invoke Converse --> Regions Invoke --> Regions Regions --> Claude Regions --> Llama Regions --> Titan Regions --> Cohere Regions --> Others

Bedrock Provider Routes

Route	API	Use Case
`bedrock/`	Default	Standard chat completions
`bedrock/converse/`	Converse API	Unified conversation interface
`bedrock/invoke/`	Invoke API	Raw model invocation
`bedrock/converse_like/`	Internal proxy	Call converse via internal proxy
`bedrock/llama/`	Llama specific	Meta Llama models
`bedrock/deepseek_r1/`	DeepSeek R1	DeepSeek on Bedrock
`bedrock/qwen3/`	Qwen3	Qwen models on Bedrock
`bedrock/openai/`	OpenAI compat	OpenAI-compatible models on Bedrock

Authentication Methods

Method	Parameters	Best For
Environment variables	`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`	Development
AWS profile	`aws_profile_name`	Local development
STS AssumeRole	`aws_role_name` (needs `sts:AssumeRole`)	Cross-account access
IAM Roles Anywhere	`aws_web_identity_token`	On-premise workloads
Session token	`aws_session_token`	Temporary credentials
External client	Pass `BedrockRuntime.Client` object	SSO sessions, assumed roles

Implementation

from dataclasses import dataclass
from typing import Optional
from enum import Enum

class BedrockRoute(Enum):
    DEFAULT = "bedrock/"
    CONVERSE = "bedrock/converse/"
    INVOKE = "bedrock/invoke/"

@dataclass
class BedrockLiteLLMGuide:
    """AWS Bedrock with LiteLLM implementation guide."""

    def get_install(self) -> str:
        """Installation and setup."""
        return (
            "# Install LiteLLM with boto3\n"
            "pip install 'litellm[proxy]'\n"
            "pip install boto3\n"
            "\n"
            "# Set AWS credentials\n"
            "export AWS_ACCESS_KEY_ID='AKIAXXX'\n"
            "export AWS_SECRET_ACCESS_KEY='xxx'\n"
            "export AWS_REGION_NAME='us-west-2'\n"
            "\n"
            "# Start proxy\n"
            "litellm --config config.yaml\n"
            "# Running on http://0.0.0.0:4000"
        )

    def get_config_yaml(self) -> str:
        """config.yaml for Bedrock models."""
        return (
            "# config.yaml\n"
            "# AWS Bedrock with LiteLLM\n"
            "\n"
            "model_list:\n"
            "  # Claude 3.5 Sonnet\n"
            "  - model_name: claude-sonnet\n"
            "    litellm_params:\n"
            "      model: bedrock/us.anthropic\n"
            "        .claude-3-5-sonnet\n"
            "        -20240620-v1:0\n"
            "      aws_access_key_id: \n"
            "        os.environ/\n"
            "          AWS_ACCESS_KEY_ID\n"
            "      aws_secret_access_key: \n"
            "        os.environ/\n"
            "          AWS_SECRET_ACCESS_KEY\n"
            "      aws_region_name: \n"
            "        os.environ/\n"
            "          AWS_REGION_NAME\n"
            "\n"
            "  # Claude 3 Haiku (fallback)\n"
            "  - model_name: claude-haiku\n"
            "    litellm_params:\n"
            "      model: bedrock/us.anthropic\n"
            "        .claude-3-haiku\n"
            "        -20240307-v1:0\n"
            "      aws_region_name: \n"
            "        os.environ/\n"
            "          AWS_REGION_NAME\n"
            "\n"
            "  # Llama 3 70B\n"
            "  - model_name: llama\n"
            "    litellm_params:\n"
            "      model: bedrock/meta.llama3\n"
            "        -70b-instruct-v1:0\n"
            "      aws_region_name: \n"
            "        os.environ/\n"
            "          AWS_REGION_NAME\n"
            "\n"
            "  # Titan Express\n"
            "  - model_name: titan\n"
            "    litellm_params:\n"
            "      model: bedrock/amazon\n"
            "        .titan-text-express-v1\n"
            "      aws_region_name: \n"
            "        os.environ/\n"
            "          AWS_REGION_NAME\n"
            "\n"
            "  # Cohere Command R\n"
            "  - model_name: cohere\n"
            "    litellm_params:\n"
            "      model: bedrock/cohere\n"
            "        .command-r-v1:0\n"
            "      aws_region_name: \n"
            "        os.environ/\n"
            "          AWS_REGION_NAME\n"
            "\n"
            "router_settings:\n"
            "  routing_strategy: \n"
            "    latency-based-routing\n"
            "  enable_pre_call_checks: true\n"
            "  fallbacks:\n"
            "    - claude-sonnet:\n"
            "        - claude-haiku\n"
            "        - llama\n"
            "\n"
            "litellm_settings:\n"
            "  num_retries: 3\n"
            "  request_timeout: 10\n"
            "  allowed_fails: 3\n"
            "  cooldown_time: 30\n"
            "\n"
            "general_settings:\n"
            "  master_key: sk-1234\n"
            "  database_url: os.environ/\n"
            "    DATABASE_URL\n"
            "  alerting: ['slack']"
        )

    def get_multi_region(self) -> str:
        """Multi-region load balancing."""
        return (
            "# config.yaml\n"
            "# Multi-region load balancing\n"
            "\n"
            "model_list:\n"
            "  # Deployment 1: us-west-2\n"
            "  - model_name: claude\n"
            "    litellm_params:\n"
            "      model: bedrock/us.anthropic\n"
            "        .claude-3-5-sonnet\n"
            "        -20240620-v1:0\n"
            "      aws_region_name: us-west-2\n"
            "      aws_access_key_id: \n"
            "        os.environ/\n"
            "          AWS_ACCESS_KEY_ID\n"
            "      aws_secret_access_key: \n"
            "        os.environ/\n"
            "          AWS_SECRET_ACCESS_KEY\n"
            "      tpm: 2000\n"
            "      rpm: 10\n"
            "\n"
            "  # Deployment 2: us-east-1\n"
            "  - model_name: claude\n"
            "    litellm_params:\n"
            "      model: bedrock/us.anthropic\n"
            "        .claude-3-5-sonnet\n"
            "        -20240620-v1:0\n"
            "      aws_region_name: us-east-1\n"
            "      aws_access_key_id: \n"
            "        os.environ/\n"
            "          AWS_ACCESS_KEY_ID\n"
            "      aws_secret_access_key: \n"
            "        os.environ/\n"
            "          AWS_SECRET_ACCESS_KEY\n"
            "      tpm: 2000\n"
            "      rpm: 10\n"
            "\n"
            "  # Deployment 3: us-east-2\n"
            "  - model_name: claude\n"
            "    litellm_params:\n"
            "      model: bedrock/us.anthropic\n"
            "        .claude-3-5-sonnet\n"
            "        -20240620-v1:0\n"
            "      aws_region_name: us-east-2\n"
            "      tpm: 2000\n"
            "      rpm: 10\n"
            "\n"
            "router_settings:\n"
            "  routing_strategy: \n"
            "    latency-based-routing\n"
            "  enable_pre_call_checks: true\n"
            "# Proxy distributes requests\n"
            "# across all three regions\n"
            "# automatically"
        )

    def get_openai_sdk_usage(self) -> str:
        """Usage with OpenAI SDK."""
        return (
            "from openai import OpenAI\n"
            "\n"
            "client = OpenAI(\n"
            "    base_url='http://localhost:'\n"
            "        '4000/v1',\n"
            "    api_key='sk-1234',\n"
            ")\n"
            "\n"
            "# Call Bedrock Claude\n"
            "response = client.chat\\\n"
            "    .completions.create(\n"
            "    model='claude-sonnet',\n"
            "    messages=[\n"
            "        {'role': 'user',\n"
            "         'content': 'Hello!'}\n"
            "    ],\n"
            ")\n"
            "print(response.choices[0]\n"
            "      .message.content)\n"
            "\n"
            "# Call Bedrock Llama\n"
            "response = client.chat\\\n"
            "    .completions.create(\n"
            "    model='llama',\n"
            "    messages=[...],\n"
            ")\n"
            "\n"
            "# Call Bedrock Titan\n"
            "response = client.chat\\\n"
            "    .completions.create(\n"
            "    model='titan',\n"
            "    messages=[...],\n"
            ")"
        )

    def get_boto3_sdk_usage(self) -> str:
        """Usage with boto3 SDK."""
        return (
            "import boto3\n"
            "import json\n"
            "import os\n"
            "\n"
            "# Set dummy AWS creds\n"
            "# (required by boto3,\n"
            "# not used by LiteLLM)\n"
            "os.environ[\n"
            "    'AWS_ACCESS_KEY_ID'] = 'dummy'\n"
            "os.environ[\n"
            "    'AWS_SECRET_ACCESS_KEY']\n"
            "    ] = 'dummy'\n"
            "os.environ[\n"
            "    'AWS_BEARER_TOKEN_BEDROCK'\n"
            "    ] = 'sk-1234'\n"
            "\n"
            "# Point boto3 to LiteLLM\n"
            "bedrock_runtime = boto3.client(\n"
            "    service_name=\n"
            "        'bedrock-runtime',\n"
            "    region_name='us-west-2',\n"
            "    endpoint_url=\n"
            "        'http://localhost:'\n"
            "        '4000/bedrock')\n"
            "\n"
            "# Call load-balanced model\n"
            "response = bedrock_runtime\\\n"
            "    .invoke_model(\n"
            "    modelId='claude',\n"
            "    contentType=\n"
            "        'application/json',\n"
            "    accept=\n"
            "        'application/json',\n"
            "    body=json.dumps({\n"
            "        'max_tokens': 100,\n"
            "        'messages': [{\n"
            "            'role': 'user',\n"
            "            'content': 'Hello!'\n"
            "        }],\n"
            "        'anthropic_version':\n"
            "            'bedrock-2023-05-31'\n"
            "    }))\n"
            "\n"
            "body = json.loads(\n"
            "    response['body'].read())\n"
            "print(body['content'][0]['text'])"
        )

    def get_sts_auth(self) -> str:
        """STS AssumeRole authentication."""
        return (
            "# config.yaml with STS AssumeRole\n"
            "\n"
            "model_list:\n"
            "  - model_name: claude\n"
            "    litellm_params:\n"
            "      model: bedrock/us.anthropic\n"
            "        .claude-3-5-sonnet\n"
            "        -20240620-v1:0\n"
            "      aws_role_name: arn:aws:iam::\n"
            "        123456789012:role/\n"
            "        BedrockAccess\n"
            "      aws_session_name: \n"
            "        litellm-session\n"
            "      aws_region_name: \n"
            "        us-west-2\n"
            "\n"
            "# Required IAM policy:\n"
            "# sts:AssumeRole on target\n"
            "# role. Error if missing:\n"
            "# 'Could not assume role'\n"
            "\n"
            "# IAM Roles Anywhere\n"
            "# (on-premise workloads):\n"
            "# aws_web_identity_token:\n"
            "#   os.environ/\n"
            "#     AWS_WEB_IDENTITY_TOKEN"
        )

    def get_fallback_config(self) -> str:
        """Fallback configuration."""
        return (
            "# config.yaml with Bedrock fallbacks\n"
            "\n"
            "model_list:\n"
            "  - model_name: claude-sonnet\n"
            "    litellm_params:\n"
            "      model: bedrock/us.anthropic\n"
            "        .claude-3-5-sonnet\n"
            "        -20240620-v1:0\n"
            "      aws_region_name: \n"
            "        $AWS_REGION\n"
            "\n"
            "  - model_name: claude-haiku\n"
            "    litellm_params:\n"
            "      model: bedrock/us.anthropic\n"
            "        .claude-3-haiku\n"
            "        -20240307-v1:0\n"
            "      aws_region_name: \n"
            "        $AWS_REGION\n"
            "\n"
            "  - model_name: llama\n"
            "    litellm_params:\n"
            "      model: bedrock/meta.llama3\n"
            "        -70b-instruct-v1:0\n"
            "      aws_region_name: \n"
            "        $AWS_REGION\n"
            "\n"
            "router_settings:\n"
            "  enable_pre_call_checks: true\n"
            "\n"
            "litellm_settings:\n"
            "  num_retries: 3\n"
            "  request_timeout: 10\n"
            "  fallbacks:\n"
            "    - claude-sonnet:\n"
            "        - claude-haiku\n"
            "        - llama\n"
            "  allowed_fails: 3\n"
            "  cooldown_time: 30\n"
            "\n"
            "# If Claude Sonnet fails:\n"
            "# 1. Retry 3 times\n"
            "# 2. Fallback to Haiku\n"
            "# 3. Fallback to Llama\n"
            "# 4. Cooldown Sonnet 30s"
        )

AWS Bedrock with LiteLLM Checklist

[ ] Install boto3: pip install boto3 (required for Bedrock requests)
[ ] Install LiteLLM: pip install 'litellm[proxy]'
[ ] Set AWS credentials as environment variables: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION_NAME
[ ] Create config.yaml with bedrock/ prefix models
[ ] Start proxy: litellm --config config.yaml (runs on localhost:4000)
[ ] Provider routes: bedrock/, bedrock/converse/, bedrock/invoke/, bedrock/converse_like/
[ ] Additional routes: bedrock/llama/, bedrock/deepseek_r1/, bedrock/qwen3/, bedrock/openai/, bedrock/moonshot/
[ ] Supported endpoints: /chat/completions, /completions, /embeddings, /images/generations, /v1/realtime, /rerank
[ ] Model format: bedrock/provider.model-name (e.g., bedrock/us.anthropic.claude-3-5-sonnet-20240620-v1:0)
[ ] Bedrock models: Claude (3.5 Sonnet, 3 Haiku, Instant), Llama (3 70B, 3 8B), Titan (Express, Embeddings)
[ ] Bedrock models: Cohere (Command R, Command Text), DeepSeek R1, Qwen3, Moonshot
[ ] Authentication: environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
[ ] Authentication: AWS profile (aws_profile_name)
[ ] Authentication: STS AssumeRole (aws_role_name, requires sts:AssumeRole permission)
[ ] Authentication: IAM Roles Anywhere (aws_web_identity_token for on-premise)
[ ] Authentication: Session token (aws_session_token for temporary credentials)
[ ] Authentication: External client (pass BedrockRuntime.Client object for SSO/assumed roles)
[ ] All auth params: aws_access_key_id, aws_secret_access_key, aws_session_token, aws_region_name, aws_session_name, aws_profile_name, aws_role_name, aws_web_identity_token, aws_bedrock_runtime_endpoint, api_key
[ ] Multi-region load balancing: same model_name, different aws_region_name
[ ] Multi-region: proxy automatically distributes requests across regions
[ ] Set tpm and rpm per deployment for rate-limit-aware routing
[ ] Routing strategies: load balancing, fallback, rate-limit-aware, latency-based
[ ] Latency-based routing: picks deployment with fastest response time
[ ] enable_pre_call_checks: true for pre-call validation
[ ] Fallbacks: claude-sonnet → claude-haiku → llama for model-level failover
[ ] Fallback config: litellm_settings: num_retries: 3, request_timeout: 10, allowed_fails: 3, cooldown_time: 30
[ ] OpenAI SDK usage: base_url=localhost:4000/v1, api_key=sk-1234, model='claude-sonnet'
[ ] Boto3 SDK usage: endpoint_url=localhost:4000/bedrock, AWS_BEARER_TOKEN_BEDROCK=sk-1234
[ ] Boto3 SDK: set dummy AWS credentials (required by boto3, not used by LiteLLM)
[ ] Boto3 SDK: modelId uses model_name from config.yaml, not actual Bedrock model ID
[ ] Invoke API: POST /bedrock/model/{model_name}/invoke with anthropic_version
[ ] Converse API: use bedrock/converse/ prefix for unified conversation interface
[ ] Replace AWS auth (AWS4-HMAC-SHA256) with Bearer LITELLM_VIRTUAL_KEY when using proxy
[ ] Spend tracking: PostgreSQL with database_url in general_settings
[ ] Virtual keys with budgets for per-team cost control
[ ] Admin dashboard at /ui for spend breakdown
[ ] Slack alerts for budget and error notifications
[ ] STS AssumeRole: IAM user/role must have sts:AssumeRole on target role
[ ] Boto3 is not async — LiteLLM handles async wrapping internally
[ ] Pass external BedrockRuntime.Client for SSO sessions or assumed role sessions
[ ] Read LLM gateway guide for gateway concepts
[ ] Read LiteLLM tutorial for proxy setup
[ ] Read fallbacks for failover strategies
[ ] Read OpenAI compatible API for unified interface
[ ] Test: Bedrock model accessible via OpenAI SDK through LiteLLM proxy
[ ] Test: multi-region load balancing distributes across regions
[ ] Test: fallback switches from Sonnet to Haiku on failure
[ ] Test: STS AssumeRole authenticates correctly
[ ] Test: boto3 SDK calls proxy with AWS_BEARER_TOKEN_BEDROCK
[ ] Test: spend tracking logs Bedrock model costs in PostgreSQL
[ ] Document config.yaml, AWS auth method, routing strategy, fallback chain, and region setup

FAQ

How do you set up LiteLLM with AWS Bedrock?

Install boto3, create config.yaml with bedrock/ prefix models, and start the proxy. LiteLLM Docs: "LiteLLM requires boto3 to be installed. Provider routes: bedrock/, bedrock/converse/, bedrock/invoke/. Supported endpoints: /chat/completions, /completions, /embeddings, /images/generations, /v1/realtime, /rerank. Setup config.yaml with model: bedrock/us.anthropic.claude-3-5-sonnet, aws_access_key_id, aws_secret_access_key, aws_region_name. Start proxy: litellm --config config.yaml." AWS Samples: "Use configuration files for routing. LiteLLM routing allows dynamic control over how requests are directed to LLMs across multiple backends." Steps: (1) pip install boto3. (2) Set AWS credentials as env vars. (3) Create config.yaml with bedrock/ models. (4) Start proxy: litellm --config config.yaml. (5) Call via OpenAI SDK at localhost:4000.

How does LiteLLM authenticate with AWS Bedrock?

LiteLLM uses boto3 for authentication, supporting all AWS credential methods. LiteLLM Docs: "LiteLLM uses boto3 to handle authentication. All options supported: aws_access_key_id, aws_secret_access_key, aws_session_token, aws_region_name, aws_session_name, aws_profile_name, aws_role_name, aws_web_identity_token, aws_bedrock_runtime_endpoint, api_key. STS AssumeRole: IAM user or role must have sts:AssumeRole permission. IAM Roles Anywhere for on-premise workloads. Pass external BedrockRuntime.Client as parameter for SSO sessions or assumed roles." Auth methods: (1) Environment variables: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY. (2) AWS profile: aws_profile_name. (3) STS AssumeRole: aws_role_name with sts:AssumeRole permission. (4) IAM Roles Anywhere: aws_web_identity_token for on-premise. (5) External client: pass BedrockRuntime.Client object. (6) Session token: aws_session_token for temporary credentials.

What Bedrock models does LiteLLM support?

LiteLLM supports all Bedrock foundation models including Claude, Llama, Titan, Cohere, and more. LiteLLM Docs: "Amazon Bedrock is a fully managed service with high-performing foundation models. Provider routes: bedrock/, bedrock/converse/, bedrock/invoke/, bedrock/converse_like/, bedrock/llama/, bedrock/deepseek_r1/, bedrock/qwen3/, bedrock/qwen2/, bedrock/openai/, bedrock/moonshot." Models: bedrock/anthropic.claude-3-5-sonnet, bedrock/anthropic.claude-3-haiku, bedrock/anthropic.claude-instant-v1, bedrock/meta.llama3-70b-instruct, bedrock/meta.llama3-8b-instruct, bedrock/amazon.titan-text-express, bedrock/cohere.command-r-v1:0, bedrock/cohere.command-text-v14. Use bedrock/converse/ for Converse API, bedrock/invoke/ for Invoke API. All accessible via same OpenAI-compatible interface.

How do you configure multi-region load balancing with Bedrock and LiteLLM?

Define multiple deployments with same model_name but different aws_region_name for automatic load balancing. LiteLLM Docs: "Define multiple deployments with same model_name for automatic load balancing. Proxy automatically distributes requests across both regions. Deployment 1: us-west-2, Deployment 2: us-east-1." AWS Samples: "Routing strategies: load balancing, fallback, rate-limit-aware routing, latency-based routing. Set tpm and rpm per deployment. enable_pre_call_checks: true for pre-call validation." Config: model_name: my-bedrock-model with two entries (us-west-2 and us-east-1). Router distributes across regions. Set tpm/rpm per deployment for rate-limit-aware routing. Use latency-based-routing for fastest response. Fallbacks: claude-3-sonnet → claude-3-haiku for model-level fallback.

How do you use the Bedrock Converse and Invoke APIs with LiteLLM?

LiteLLM supports both Converse API (bedrock/converse/) and Invoke API (bedrock/invoke/) endpoints. LiteLLM Docs: "Call Bedrock invoke endpoint through LiteLLM Proxy. POST to /bedrock/model/my-bedrock-model/invoke with Authorization: Bearer sk-1234. Body: max_tokens, messages, anthropic_version. Load balancing: define multiple deployments with same model_name, different regions." LiteLLM Docs: "Provider routes: bedrock/converse/ for Converse API, bedrock/invoke/ for Invoke API, bedrock/converse_like/ for internal proxy. Use boto3 SDK: point bedrock_runtime to localhost:4000/bedrock with AWS_BEARER_TOKEN_BEDROCK as LiteLLM key. Replace AWS auth with Bearer LITELLM_VIRTUAL_KEY." Invoke: POST /bedrock/model/{model_name}/invoke. Converse: use bedrock/converse/ prefix. Boto3 SDK: endpoint_url=http://localhost:4000/bedrock, AWS_BEARER_TOKEN_BEDROCK=sk-1234.

Want a self-hosted AI company brain that does all of this out of the box?
Book a demo →