How to Choose Between LangChain, LlamaIndex, and Raw API Calls in 2026

By Fadi Updated Apr 19, 2026 14 min read Dev Fundamentals

TL;DR — Default to raw API, reach for framework when you hit its wall. Sabaoon: "RAG → LlamaIndex. Multi-step agents → LangChain. Everything else → raw SDK. Default in 2026: start raw, add framework only when you hit a wall it was designed to solve." CodeNicely: "1-2 LLM calls: raw API. Multi-step agents: LangGraph. Middle case (3-4 calls): 200-line orchestrator is clearer. Abstractions leak under edge cases — you pay the tax without escaping complexity." Ranksquire: "ATR: LangChain 1.45, LlamaIndex 1.12, raw API 0.38. LlamaIndex: 6ms overhead, 30-40% less code for RAG. Neither framework right for deterministic batch tasks." Markaicode: "Raw API wins every metric: P95 0.9s vs 1.8s, 153MB vs 412MB, 12 vs 87 dependencies. LangChain adds 200ms overhead per call." Learn more with RAG vs fine-tuning, prompt engineering, build AI feature, and choose LLM.

Sabaoon frames the 2026 landscape: "When LangChain launched in 2022, it became the default choice for building LLM-powered applications. Then LlamaIndex emerged as the go-to for retrieval-augmented generation. In 2026, many production teams are quietly abandoning both in favor of thin wrappers around the raw Anthropic and OpenAI SDKs."

CodeNicely provides the honest answer: "The token overhead and learning curve are not the real cost of these frameworks. The real cost is that their abstractions leak under edge cases in ways that force you to understand the underlying API anyway. You end up paying the abstraction tax without escaping the raw-API complexity."

Framework Decision Architecture

flowchart TD subgraph Decision["Decision Tree"] Q1{"Is your app
primarily RAG
(Q&A over documents)?"} Q2{"Do you need complex
multi-step chains with
many pre-built integrations?"} Q3{"Is it 1-2 LLM calls
with clear I/O?"} Q4{"More than 3 distinct
LLM features or
multi-provider needed?"} end subgraph LangChain["LangChain / LangGraph"] LC_Chains["Chains
sequential LLM calls
prompt chaining
memory management"] LC_Agents["Agents
tool-calling loops
state management
checkpointing
human approval"] LC_Integrations["Integrations
dozens of services
vector stores
document loaders
LLM providers"] LC_Cost["Cost
ATR: 1.45
200ms overhead/call
87 dependencies
412MB memory
frequent breaking changes"] end subgraph LlamaIndex["LlamaIndex"] LI_Index["Index
document loading
chunking strategies
embedding and vector storage
queryable representation"] LI_Retrieval["Retrieval
query decomposition
re-ranking
hybrid retrieval
context precision"] LI_RAG["RAG Pipeline
30-40% less code
than LangChain
6ms overhead
purpose-built for retrieval"] LI_Cost["Cost
ATR: 1.12
34 dependencies
287MB memory
$70/mo self-hosted
$500/mo LlamaCloud Pro"] end subgraph RawAPI["Raw API + Custom Layer"] Raw_Calls["Direct Calls
httpx or aiohttp
no abstraction layers
full control
you own the code"] Raw_Perf["Performance
P95: 0.9s
153MB memory
12 dependencies
no framework overhead"] Raw_Cost["Cost
ATR: 0.38
immune to version churn
smallest bundle
fastest debugging
code you wrote"] end subgraph Use["Best Use Cases"] RAG_Use["RAG / Document Q&A
enterprise knowledge bases
complex retrieval pipelines"] Agent_Use["Multi-step Agents
tool use
stateful workflows
orchestration"] Simple_Use["Simple Calls
classification
extraction
summarization
single-turn chat"] Batch_Use["Batch Processing
deterministic tasks
high-volume
non-reasoning"] end Q1 -->|Yes| LlamaIndex Q1 -->|No| Q2 Q2 -->|Yes| LangChain Q2 -->|No| Q3 Q3 -->|Yes| RawAPI Q3 -->|No| Q4 Q4 -->|Yes| LangChain Q4 -->|No| RawAPI LlamaIndex --> RAG_Use LangChain --> Agent_Use RawAPI --> Simple_Use RawAPI --> Batch_Use style LangChain fill:#4169E1,color:#fff style LlamaIndex fill:#39FF14,color:#000 style RawAPI fill:#2D1B69,color:#fff

Framework Performance Comparison

Metric	LangChain (0.2.0)	LlamaIndex (0.10.0)	Raw API + httpx
Query P95 latency	1.8s	1.2s	0.9s
Initial index time	42.3s	38.7s	36.1s
Memory overhead	412MB	287MB	153MB
Dependencies	87 packages	34 packages	12 packages
ATR (debug/build)	1.45	1.12	0.38
Per-call overhead	~200ms	~6ms	0ms
API stability	Frequent breaks	Stable	Stable
Bundle size	Large	Moderate	Minimal
Debugging	Hard (layers)	Moderate	Easy (your code)

Implementation

from dataclasses import dataclass
from typing import Optional
from enum import Enum

class FrameworkChoice(Enum):
    LANGCHAIN = "langchain"
    LLAMAINDEX = "llamaindex"
    RAW_API = "raw_api"

@dataclass
class FrameworkComparisonGuide:
    """LangChain vs LlamaIndex vs raw API guide."""

    def get_raw_api(self) -> str:
        """Raw API implementation."""
        return (
            "# === RAW API IMPLEMENTATION ===\n"
            "import httpx, json, time\n"
            "from dataclasses import dataclass\n"
            "\n"
            "@dataclass\n"
            "class LLMResponse:\n"
            "    content: str\n"
            "    tokens: int\n"
            "    latency_ms: float\n"
            "\n"
            "class RawLLMClient:\n"
            "    '''Direct API calls, no framework.''' \n"
            "    def __init__(self,\n"
            "                 model='llama3.2'):\n"
            "        self.model = model\n"
            "        self.base_url = (\n"
            "            'http://localhost:11434')\n"
            "    \n"
            "    async def chat(\n"
            "        self,\n"
            "        prompt: str,\n"
            "        system: str = None,\n"
            "        temperature: float = 0.3,\n"
            "        max_tokens: int = 1000\n"
            "    ) -> LLMResponse:\n"
            "        '''Single LLM call.''' \n"
            "        messages = []\n"
            "        if system:\n"
            "            messages.append({\n"
            "                'role': 'system',\n"
            "                'content': system})\n"
            "        messages.append({\n"
            "            'role': 'user',\n"
            "            'content': prompt})\n"
            "        \n"
            "        start = time.time()\n"
            "        async with httpx.AsyncClient(\n"
            "            timeout=300.0\n"
            "        ) as client:\n"
            "            resp = await (\n"
            "                client.post(\n"
            "                    f'{self.base_url}'\n"
            "                    '/api/chat',\n"
            "                    json={\n"
            "                        'model': self.model,\n"
            "                        'messages': messages,\n"
            "                        'options': {\n"
            "                            'temperature': (\n"
            "                              temperature),\n"
            "                            'num_predict': (\n"
            "                              max_tokens)}}))\n"
            "            data = resp.json()\n"
            "            return LLMResponse(\n"
            "                content=data['message']\n"
            "                  ['content'],\n"
            "                tokens=data.get(\n"
            "                    'eval_count', 0),\n"
            "                latency_ms=(\n"
            "                    (time.time() - start)\n"
            "                    * 1000))\n"
            "    \n"
            "    async def classify(\n"
            "        self, text: str) -> str:\n"
            "        '''Simple classification.''' \n"
            "        resp = await self.chat(\n"
            "            prompt=f'Classify: {text}',\n"
            "            system='Return one word:',\n"
            "              ' positive, negative,'\n"
            "              ' or neutral.',\n"
            "            temperature=0,\n"
            "            max_tokens=10)\n"
            "        return resp.content.strip()\n"
            "    \n"
            "    async def extract(\n"
            "        self, text: str) -> dict:\n"
            "        '''Structured extraction.''' \n"
            "        resp = await self.chat(\n"
            "            prompt=f'Extract from: {text}',\n"
            "            system='Return JSON only.',\n"
            "            temperature=0,\n"
            "            max_tokens=500)\n"
            "        return json.loads(\n"
            "            resp.content)\n"
            "\n"
            "# 12 dependencies, 153MB,\n"
            "#   P95 0.9s, ATR 0.38\n"
            "# No framework overhead\n"
            "# Full control, easy debug"
        )

    def get_rag_comparison(self) -> str:
        """RAG implementation comparison."""
        return (
            "# === RAG: 3 APPROACHES ===\n"
            "\n"
            "# --- LlamaIndex (best for RAG) ---\n"
            "# pip install llamaindex\n"
            "'''\n"
            "from llama_index.core import (\n"
            "    VectorStoreIndex,\n"
            "    SimpleDirectoryReader)\n"
            "\n"
            "documents = (\n"
            "    SimpleDirectoryReader(\n"
            "        './docs').load_data())\n"
            "index = VectorStoreIndex.from_documents(\n"
            "    documents)\n"
            "query_engine = index.as_query_engine()\n"
            "response = query_engine.query(\n"
            "    'What is RAG?')\n"
            "'''\n"
            "# 34 deps, 287MB, P95 1.2s\n"
            "# 30-40% less code than LangChain\n"
            "# Purpose-built for retrieval\n"
            "\n"
            "# --- LangChain (overkill for RAG) ---\n"
            "# pip install langchain langchain-community\n"
            "'''\n"
            "from langchain_community.document_loaders import (\n"
            "    DirectoryLoader)\n"
            "from langchain_text_splitters import (\n"
            "    RecursiveCharacterTextSplitter)\n"
            "from langchain_community.embeddings import (\n"
            "    OllamaEmbeddings)\n"
            "from langchain_community.vectorstores import (\n"
            "    Chroma)\n"
            "from langchain_community.llms import Ollama\n"
            "from langchain.chains import (\n"
            "    RetrievalQA)\n"
            "\n"
            "loader = DirectoryLoader('./docs')\n"
            "docs = loader.load()\n"
            "splitter = (\n"
            "    RecursiveCharacterTextSplitter())\n"
            "chunks = splitter.split_documents(docs)\n"
            "embeddings = OllamaEmbeddings()\n"
            "vectorstore = Chroma.from_documents(\n"
            "    chunks, embeddings)\n"
            "llm = Ollama(model='llama3.2')\n"
            "qa = RetrievalQA.from_chain_type(\n"
            "    llm=llm,\n"
            "    retriever=vectorstore\n"
            "      .as_retriever())\n"
            "response = qa.invoke(\n"
            "    {'query': 'What is RAG?'})\n"
            "'''\n"
            "# 87 deps, 412MB, P95 1.8s\n"
            "# 200ms overhead per call\n"
            "# Frequent breaking changes\n"
            "\n"
            "# --- Raw API (manual but fast) ---\n"
            "'''\n"
            "import httpx, json, asyncio\n"
            "\n"
            "async def rag_query(\n"
            "    query: str,\n"
            "    docs: list[str]) -> str:\n"
            "    # 1. Embed query\n"
            "    # 2. Search vector DB\n"
            "    # 3. Build prompt with context\n"
            "    # 4. Call LLM directly\n"
            "    context = '\\n'.join(docs[:5])\n"
            "    prompt = f'Context: {context}'\n"
            "      f'\\n\\nQuery: {query}'\n"
            "    # ... call Ollama directly\n"
            "    return response\n"
            "'''\n"
            "# 12 deps, 153MB, P95 0.9s\n"
            "# Manual but full control\n"
            "# Best for simple retrieval"
        )

    def get_decision_matrix(self) -> dict:
        """Decision matrix."""
        return {
            "rag_docs": "RAG over documents → LlamaIndex. Purpose-built, 30-40% less code, 6ms overhead.",
            "agents": "Multi-step agents, tool use → LangChain/LangGraph. State management, checkpointing, integrations.",
            "simple_calls": "1-2 LLM calls with clear I/O → raw API. Classification, extraction, summarization, single-turn chat.",
            "middle_case": "3-4 sequential calls with simple branching → 200-line custom orchestrator. Clearer than LangChain.",
            "batch": "Deterministic, high-volume, non-reasoning → raw API. Frameworks add overhead with no value.",
            "serverless": "Serverless deployment → raw API. Cold starts matter, package size matters.",
            "multi_provider": "Multi-provider support needed → LangChain or custom abstraction layer. Raw API if provider fixed.",
            "debug_priority": "Debugging is priority → raw API. Bug is in code you wrote, not framework layers.",
            "performance": "Sub-100ms requirements → raw API. Frameworks add latency making this difficult.",
            "default_2026": "Default in 2026: start with raw SDK. Reach for framework only when you hit a wall it was designed to solve.",
        }

Framework Choice Checklist

[ ] Default in 2026: start with raw SDK, reach for framework only when you hit a wall it was designed to solve
[ ] Is your app primarily RAG (Q&A over documents)? → LlamaIndex
[ ] Do you need complex multi-step chains with many pre-built integrations? → LangChain (accept maintenance cost)
[ ] Is it 1-2 LLM calls with clear inputs and outputs? → raw API (classification, extraction, summarization, single-turn chat)
[ ] 3-4 sequential calls with simple branching? → 200-line custom orchestrator is clearer than LangChain
[ ] More than 3 distinct LLM features or multi-provider support needed? → LangChain or custom abstraction
[ ] Deterministic, high-volume, non-reasoning batch tasks? → raw API (frameworks add overhead with no value)
[ ] Serverless deployment? → raw API (cold starts matter, package size matters)
[ ] Sub-100ms latency requirements? → raw API (frameworks add latency making this difficult)
[ ] Debugging is top priority? → raw API (bug is in code you wrote, not framework layers)
[ ] LangChain: chains, agents, memory, tool abstractions connecting LLM calls into complex pipelines
[ ] LangChain is now split into langchain-core, langchain-community, and provider-specific packages
[ ] LangChain/LangGraph for orchestration: agents, tool use, multi-step workflows, state management, checkpointing
[ ] LangChain costs: ATR 1.45 (1.45h debugging per 1h building), 200ms overhead per call, 87 dependencies, 412MB memory
[ ] LangChain risks: abstraction leakage, version instability (frequent breaking changes), overhead for simple use cases
[ ] LlamaIndex: specializes in document indexing and retrieval-augmented generation (RAG)
[ ] LlamaIndex core abstraction is the Index — queryable representation of data with chunking, embedding, vector storage
[ ] LlamaIndex for RAG: 30-40% less code than LangChain, 6ms overhead per request, purpose-built for retrieval
[ ] LlamaIndex costs: ATR 1.12, 34 dependencies, 287MB memory, $70/mo self-hosted, $500/mo LlamaCloud Pro
[ ] LlamaIndex crossover at ~7,000 queries per day for self-hosted vs managed
[ ] LlamaIndex is wrong for: simple retrieval (pgvector + 50 lines is better), general agent framework (not optimized for that)
[ ] Raw API: call LLM provider SDK directly, build your own pipeline logic without framework
[ ] Raw API wins on every metric: P95 0.9s vs 1.8s, 153MB vs 412MB, 12 vs 87 dependencies
[ ] Raw API ATR 0.38 — lowest debugging cost, immune to framework version churn
[ ] Raw API is right when: 1-2 LLM calls, deterministic performance needed, serverless, you understand underlying API
[ ] Raw API is wrong when: more than 3 distinct LLM features, multi-provider needed, agent-style tool use required
[ ] Abstraction Tax Ratio (ATR): LangChain 1.45, LlamaIndex 1.12, raw API 0.38
[ ] Every framework you add becomes a dependency you maintain — its bugs, breaking changes, and community opinions
[ ] Abstractions leak under edge cases — you pay the tax without escaping raw-API complexity
[ ] For simple cases, raw SDK is less code and more readable than any framework
[ ] LlamaIndex optimizes for data quality at retrieval layer — chunking, indexing, query decomposition, re-ranking
[ ] LangGraph optimizes for reliability at orchestration layer — state management, checkpointing, graph execution, human approval
[ ] Use each framework for its center of gravity: LlamaIndex for retrieval, LangGraph for agent behavior
[ ] Measure your ATR before adding either framework to a batch pipeline
[ ] Take one LangChain chain in your codebase, rewrite with raw API calls, measure performance difference
[ ] Read RAG vs fine-tuning for knowledge approach
[ ] Read prompt engineering for prompt design
[ ] Read build AI feature for architecture selection
[ ] Read choose LLM for model selection
[ ] Test: raw API implementation matches framework output quality
[ ] Test: P95 latency meets requirement with chosen approach
[ ] Test: dependency count and bundle size within constraints
[ ] Test: debugging time (ATR) is acceptable for team
[ ] Test: framework version upgrades don't break production
[ ] Test: abstraction leakage doesn't force reading framework source
[ ] Document framework choice, ATR measurement, performance benchmarks, decision rationale, escape plan

FAQ

When should you use LangChain vs LlamaIndex vs raw API calls?

LangChain for agents and orchestration, LlamaIndex for RAG and retrieval, raw API for simple calls. Sabaoon: "Decision tree: is your app primarily RAG (Q&A over documents)? YES → LlamaIndex. NO. Do you need complex multi-step chains with many pre-built integrations? YES → LangChain (but accept maintenance cost). NO → Raw SDK + custom utility layer. The default in 2026 should be to start with the raw SDK and reach for a framework only when you hit a wall it was specifically designed to solve." CodeNicely: "If your feature is one or two LLM calls with well-defined inputs and outputs, call the API directly. If you have multi-step workflows, tool use, or stateful agents, LangGraph earns its weight. The middle case — three or four sequential calls with simple branching — is where teams over-adopt LangChain. A 200-line orchestrator is usually clearer." DEV Community: "Do not default to LangChain because it is popular. Start with raw API calls. Reach for LlamaIndex when your problem is fundamentally about retrieval. Use LangChain when you need a stateful multi-step agent loop." When: (1) RAG over documents → LlamaIndex. (2) Multi-step agents, tool use → LangChain/LangGraph. (3) Simple calls (1-2 LLM) → raw API. (4) 3-4 sequential calls → 200-line custom orchestrator. (5) Default: start raw, add framework when you hit its specific wall.

What is the abstraction tax and how does it affect LLM applications?

Abstraction Tax Ratio measures debugging hours vs implementation hours. Ranksquire: "ATR = Total Engineering Debug Hours / Total Feature Implementation Hours. LangChain teams in production: ATR ~1.45 — for every hour building features, 1.45 hours debugging framework abstractions. LlamaIndex teams: ~1.12. Raw API teams: ~0.38. Neither framework is the right choice for deterministic, non-reasoning batch tasks." Markaicode: "LangChain ChatOpenAI call with streaming adds ~200ms overhead on local testing — before any chains or agents. For high-volume applications, that is unacceptable. Framework overhead can erase Python 3.12 performance gains entirely." Sabaoon: "Abstraction leakage: when something goes wrong, debugging through LangChain layers is painful. Version instability: API has changed significantly multiple times. Overhead: for simple use cases, importing thousands of lines for a single chat() call." Abstraction tax: (1) LangChain ATR 1.45: 1.45h debugging per 1h building. (2) LlamaIndex ATR 1.12. (3) Raw API ATR 0.38. (4) LangChain adds ~200ms overhead per call. (5) 87 transitive dependencies vs 12 for raw API.

How do LangChain, LlamaIndex, and raw API compare on performance?

Raw API wins on every metric: latency, memory, dependencies. Markaicode: "Benchmarks: LangChain (0.2.0) — initial index 42.3s, query P95 1.8s, memory 412MB, 87 packages. LlamaIndex (0.10.0) — 38.7s, 1.2s, 287MB, 34 packages. Raw API + httpx — 36.1s, 0.9s, 153MB, 12 packages. Raw API wins on every metric because it does exactly what is needed — no more." Ranksquire: "LlamaIndex adds ~6ms overhead per request and requires 30-40% less code than LangChain for equivalent RAG pipelines. Self-hosting costs ~$70/month. Managed LlamaCloud Pro: $500/month. Crossover at ~7,000 queries per day." Sabaoon: "For simple cases, the raw SDK is less code and more readable. LangChain debugging is hard through abstraction layers. Bundle size is large. API has frequent breaking changes." Performance: (1) Query P95: raw 0.9s, LlamaIndex 1.2s, LangChain 1.8s. (2) Memory: raw 153MB, LlamaIndex 287MB, LangChain 412MB. (3) Dependencies: raw 12, LlamaIndex 34, LangChain 87. (4) LlamaIndex: 6ms overhead, 30-40% less code than LangChain for RAG.

Is LlamaIndex worth it for RAG applications?

Yes, for RAG-specific workloads — its retrieval abstractions genuinely save implementation effort. Sabaoon: "LlamaIndex specializes in document indexing and RAG. If your application needs to answer questions over a corpus of documents, LlamaIndex abstractions genuinely save significant implementation effort. Implementing from scratch with raw SDK would require significant work. For RAG specifically, LlamaIndex abstractions are worth the dependency." CodeNicely: "LlamaIndex is wrong when your retrieval is simple — a few hundred docs, flat structure. pgvector + 50-line query function will outperform it on debuggability. Also wrong as general agent framework — not what it is optimized for." Ranksquire: "LlamaIndex optimizes for data quality at retrieval layer. Every feature — chunking strategies, index types, query decomposition, re-ranking — designed to make information your LLM receives more accurate." LlamaIndex for RAG: (1) Purpose-built for document indexing and retrieval. (2) 30-40% less code than LangChain for RAG. (3) 6ms overhead per request. (4) Wrong for simple retrieval: pgvector + 50 lines is better. (5) Wrong for agents: not optimized for that. (6) Right for: enterprise knowledge bases, document Q&A, complex retrieval.

When should you avoid frameworks and use raw API calls instead?

For simple calls, deterministic tasks, serverless, and when you understand the underlying API. Markaicode: "Use raw API when: you are calling one API endpoint (just use httpx or aiohttp). You need deterministic performance (framework middleware is unpredictable). You are deploying to serverless (cold starts matter, package size matters). You actually understand what is happening (abstractions you do not understand are liabilities)." CodeNicely: "Raw API is right when the feature is one or two LLM calls with clear inputs and outputs — classification, extraction, summarization, single-turn chat. Also right when you have already chosen your provider and do not expect to switch. Raw API is wrong when you will have more than ~3 distinct LLM features, multi-provider support is a real requirement, or you need agent-style tool use." DEV Community: "Start with raw API calls — they are simpler, faster to debug, and immune to framework version churn. The abstraction tax is real. Every framework you add becomes a dependency you maintain — including its bugs, its breaking changes, and its community opinions about your architecture." Avoid frameworks when: (1) One or two LLM calls with clear I/O. (2) Deterministic performance needed. (3) Serverless deployment (cold starts, package size). (4) You understand the underlying API. (5) Classification, extraction, summarization, single-turn chat. (6) Provider already chosen, no switching expected.

Want a self-hosted AI company brain that does all of this out of the box?
Book a demo →