AWS Bedrock AI Deployment: Enterprise Guide to Managed RAG

TL;DR — AWS Bedrock is a fully managed AI service providing access to foundation models (Claude, Llama, Nova, Mistral) through a single API. Managed Knowledge Base (GA June 2026) offers fully managed RAG with 6 data source connectors, Smart Parsing, agentic retrieval, document-level ACLs, and multimodal support. AgentCore provides agent runtime with observability. Guardrails handle content filtering and PII masking. Pricing: $0.06-15/M tokens depending on model. Use Bedrock when: already on AWS, need fast time-to-value (2-4 weeks), need managed compliance (HIPAA, FedRAMP), or <500K queries/month. Use self-hosted AI when: data sovereignty required, 500K+ queries/month (cheaper), need multi-cloud portability, or need air-gapped deployment. Hybrid approach: use LiteLLM to route 80% of queries to self-hosted Llama 3 ($0.11/M) and 20% to Bedrock for complex reasoning — saves 60-80% vs pure Bedrock. Avoid vendor lock-in by keeping business logic model-independent.

AWS Bedrock Managed Knowledge Base became generally available in June 2026, offering a fully managed RAG service with data source connectors, vector storage, embeddings, reranking, and agentic retrieval — all without managing infrastructure.

For AWS-native teams, Bedrock provides the fastest path to production AI. For teams needing data sovereignty, multi-cloud portability, or cost optimization at scale, self-hosted AI remains the better choice. This post covers both paths and when to choose each.

AWS Bedrock Architecture

Bedrock Models and Pricing

Managed Knowledge Base Features

Bedrock vs Self-Hosted AI

Hybrid Deployment: Best of Both Worlds

class HybridAIGateway:
    """Route queries between AWS Bedrock and self-hosted models."""

    def __init__(self):
        self.router = LiteLLMRouter(
            providers={
                # Self-hosted for 80% of traffic (high-volume, simple)
                "local": {
                    "model": "ollama/llama3:70b",
                    "api_base": "http://ollama:11434",
                },
                # Bedrock for 20% of traffic (complex, frontier)
                "bedrock": {
                    "model": "bedrock/anthropic.claude-3-5-sonnet",
                    "aws_region": "us-east-1",
                },
                # Bedrock Haiku for fast, cheap tasks
                "bedrock_haiku": {
                    "model": "bedrock/anthropic.claude-3-haiku",
                    "aws_region": "us-east-1",
                },
            },
            routing_strategy="cost_aware",
        )

    def answer(self, query: str, complexity: str = "auto") -> dict:
        if complexity == "auto":
            complexity = self._assess_complexity(query)

        if complexity == "simple":
            # Route to self-hosted Llama 3 ($0.11/M tokens)
            model = "local"
        elif complexity == "medium":
            # Route to Bedrock Haiku ($0.25/M input)
            model = "bedrock_haiku"
        else:
            # Route to Bedrock Claude Sonnet ($3/M input)
            model = "bedrock"

        response = self.router.completion(
            model=model,
            messages=[{"role": "user", "content": query}],
        )

        return {"answer": response.choices[0].message.content, "model": model}

Cost Comparison by Scale

Hybrid assumes 80% self-hosted ($0.11/M), 15% Bedrock Haiku ($0.25/M), 5% Bedrock Sonnet ($3/M).

Bedrock Guardrails

AWS Bedrock Deployment Checklist

• [ ] Enable model access in Bedrock console (per model, per region)
• [ ] Choose deployment type: Managed KB (fast) or customer-managed (control)
• [ ] Configure data source connectors (S3, SharePoint, Confluence, Google Drive)
• [ ] Set up document-level ACLs for permissions
• [ ] Configure embedding model (Titan Text Embeddings V2 or custom)
• [ ] Configure reranking model for cross-encoder reranking
• [ ] Enable Smart Parsing for multimodal documents
• [ ] Set up guardrails: content filtering, PII masking, prompt injection detection
• [ ] Configure AgentCore for agent workflows with observability
• [ ] Set up VPC endpoints for private network access
• [ ] Configure CloudWatch monitoring for latency, errors, token usage
• [ ] Enable CloudTrail for audit logging (EU AI Act compliance)
• [ ] Set up IAM roles with least-privilege access
• [ ] Choose region for data residency compliance
• [ ] Estimate monthly costs based on expected query volume
• [ ] Compare costs with self-hosted AI TCO
• [ ] Consider hybrid deployment with LiteLLM for cost optimization
• [ ] Test with semantic answer caching to reduce API calls
• [ ] Evaluate vendor lock-in risk — keep business logic model-independent
• [ ] Consider AI company brain integration
• [ ] Use query rewriting for complex queries before Bedrock
• [ ] Set up Bedrock Guardrails for hallucination prevention
• [ ] Monitor spend: use AWS Cost Explorer for per-model cost breakdown
• [ ] Consider provisioned capacity for predictable high-volume workloads
• [ ] Test failover: what happens if Bedrock is unavailable? (fallback to self-hosted)
• [ ] Review Bedrock's compliance certifications for your industry
• [ ] Document architecture for self-hosted AI migration if needed

FAQ

What is AWS Bedrock?

AWS Bedrock is a fully managed service that provides access to foundation models from Amazon, Anthropic, Meta, Cohere, and others through a single API. It includes Managed Knowledge Base (fully managed RAG with data source connectors, vector storage, embeddings, reranking, and agentic retrieval), AgentCore (agent runtime with observability), guardrails (content filtering, PII masking), and model evaluation tools. Bedrock handles infrastructure, scaling, and compliance (HIPAA, SOC 2, FedRAMP). You pay per-token for inference. Available in 8+ AWS regions including GovCloud.

When should I use AWS Bedrock vs self-hosted AI?

Use AWS Bedrock when: you're already on AWS, you want fast time-to-value (2-4 weeks vs 2-4 months for self-hosted), you need managed compliance (HIPAA, FedRAMP, SOC 2), you have <500K queries/month (below self-hosting break-even), or your team is platform-light. Use self-hosted AI when: you need data sovereignty (data cannot leave your network), you have 500K+ queries/month (self-hosting is cheaper), you need multi-cloud portability, you want full control over retrieval and generation, or you need air-gapped deployment. Many enterprises use hybrid: Bedrock for prototyping and specialized tasks, self-hosted for high-volume RAG.

How much does AWS Bedrock cost?

AWS Bedrock costs per million tokens, varying by model. Claude 3.5 Sonnet: $3/M input, $15/M output. Claude 3 Haiku: $0.25/M input, $1.25/M output. Amazon Nova Lite 2: $0.06/M input, $0.24/M output. Llama 3 70B: $0.72/M input, $0.72/M output. Managed Knowledge Base adds vector storage costs (~$0.10/GB/month) and embedding costs. For 500K queries/month on Claude Sonnet, expect ~$2,000-4,000/month. Compare to self-hosted Llama 70B at ~$2,000-4,000/month infrastructure — break-even at ~500K queries.

Does AWS Bedrock support RAG?

Yes. Bedrock Managed Knowledge Base (GA June 2026) is a fully managed RAG service with: 6 native data source connectors (S3, SharePoint, Confluence, Google Drive, OneDrive, Web Crawler), automatic data syncing, managed vector storage, Smart Parsing (auto-selects parsing strategy per document type), hybrid search, document ranking, agentic retrieval (multi-hop reasoning with query decomposition), document-level permissions via ACLs, and multimodal support (text, images, audio, video). You can also use customer-managed Knowledge Base with OpenSearch Serverless, Aurora PostgreSQL, or Amazon Neptune for full control.

Can I use AWS Bedrock with self-hosted models?

Yes, through a hybrid deployment. Use LiteLLM as a provider-agnostic gateway to route queries between Bedrock (for frontier models like Claude) and self-hosted Ollama/vLLM (for Llama 3 at high volume). Route 80% of queries to self-hosted models ($0.11/M tokens) and 20% to Bedrock for complex reasoning. This hybrid approach saves 60-80% vs pure Bedrock while maintaining access to frontier models. Use Bedrock's guardrails for content filtering on all queries, regardless of which model serves them.

Want a self-hosted AI company brain that does all of this out of the box?
Book a demo →