RAG Chunking Strategies: 7 Methods Compared with Benchmarks
TL;DR — RAG chunking strategies determine whether the model sees coherent passages or sliced-up fragments. Vecta: "Recursive 512 won 69% accuracy. Fixed 512 close behind at 67%. Semantic chunking only 54% with imperfect tuning — it requires careful threshold calibration. Recursive splitting produced strong results with default parameters." Firecrawl: "Start with RecursiveCharacterTextSplitter at 400-512 tokens with 10-20% overlap. Page-level won NVIDIA benchmarks (0.648 accuracy) for PDFs. Semantic improves recall up to 9% but costs more." NeuroLink: "Start with recursive, measure, then optimize. Recursive handles 80% of use cases. Overlap is not free — start with zero and add only if retrieval fails at boundaries." TeachMeIDEA: "Format-aware splitting is the single biggest retrieval-quality improvement — use MarkdownHeaderTextSplitter for docs, from_language for code." arxiv: "Semantic chunking aligns with thematic coherence but incurs high computational cost and threshold selection challenges." Learn more with what is RAG, how RAG works, build from scratch, and chunk size.
Vecta summarizes their benchmark: "We benchmarked 7 chunking strategies on real-world data. Most 'best practice' advice was wrong for us. LangChain's RecursiveCharacterTextSplitter at 512 tokens achieved the highest accuracy (69%) across all seven strategies. Fixed 512 was close behind at 67%. Semantic chunking produced 17,481 chunks averaging 43 tokens — only 54% accuracy and 0.42 document F1. The brittleness of semantic chunking under imperfect tuning is itself a finding."
TeachMeIDEA frames the stakes: "If your retrieval-augmented generation system surfaces documents that contain the right keywords but miss the actual answer, your chunking step is usually the culprit. RAG chunking strategies decide whether the model sees a coherent passage or a sliced-up paragraph that loses its meaning halfway through."
Chunking Strategy Decision Architecture
is the document?"} Q1 -->|"Markdown"| MD["Markdown Header Splitter
(split on H1/H2/H3)"] Q1 -->|"Code"| Code["Code Splitter
(class/function boundaries)"] Q1 -->|"HTML"| HTML["HTML Splitter
(DOM-aligned)"] Q1 -->|"PDF/Paginated"| Page["Page-Level Chunking
(NVIDIA benchmark winner)"] Q1 -->|"Generic text"| Q2{"High-value corpus
with topic shifts?"} Q2 -->|"Yes, budget allows"| Semantic["Semantic Chunking
(embedding-based boundaries)"] Q2 -->|"No"| Recursive["Recursive Character Split
(512 tokens, 10-20% overlap)"] MD --> Q3{"Sections too
short/fragmented?"} Q3 -->|"Yes"| SemMD["Semantic Markdown
(section + topic aware)"] Q3 -->|"No"| Store["Store chunks with metadata"] Code --> Store HTML --> Store Page --> Store Semantic --> Store Recursive --> Store SemMD --> Store Store --> Eval["Evaluate: recall@k,
precision@k, RAGAS"] Eval --> Good{"Metrics good?"} Good -->|"Yes"| Done["Production Ready"] Good -->|"No"| Adjust["Adjust strategy,
size, or overlap"] Adjust --> Store
7 Chunking Strategies Benchmark
| Strategy | Chunk Size | k | Accuracy | Doc F1 | Page F1 | Groundedness | Speed |
|---|---|---|---|---|---|---|---|
| Recursive 512 | 512 tok | 5 | 69% | 0.86 | 0.92 | 0.81 | Fast |
| Fixed 512 | 512 tok | 5 | 67% | 0.85 | 0.88 | 0.85 | Fastest |
| Fixed 1024 | 1024 tok | 3 | 61% | 0.88 | 0.72 | 0.86 | Fast |
| Doc-Structure | ≤1024 tok | 2 | 52% | 0.88 | 0.69 | 0.84 | Fast |
| Page-per-Chunk | per page | 2 | 57% | 0.88 | 0.69 | 0.81 | Fast |
| Semantic | ~43 tok avg | 46 | 54% | 0.42 | 0.91 | 0.81 | Slow |
| Proposition | per proposition | 115 | 51% | 0.27 | 0.97 | 0.87 | Slow |
Chunking Strategy Selection Guide
| Your Situation | Use This Strategy | Why |
|---|---|---|
| General documents, unknown format | Recursive 512 | 69% accuracy, handles 80% of cases |
| Uniform documents, prototypes | Fixed-size 512 | Fastest, simplest, 67% accuracy |
| PDFs and paginated documents | Page-level | NVIDIA benchmark winner (0.648 accuracy) |
| Markdown documentation | Markdown header splitter | Section-aligned, preserves structure |
| Source code | Recursive with code separators | Respects function/class boundaries |
| High-value corpus, topic shifts | Semantic chunking | Up to 9% better recall, but needs tuning |
| Short-form content (Q&A, tweets) | Sentence-based | Preserves complete thoughts |
| Strict token budget | Token-based | Token-precise, matches embedding model |
| Academic papers (LaTeX) | LaTeX splitter | Section-aligned for equations and proofs |
Implementation
import hashlib
from dataclasses import dataclass
from typing import Optional
from enum import Enum
class ChunkingStrategy(Enum):
FIXED_SIZE = "fixed_size"
RECURSIVE = "recursive"
SEMANTIC = "semantic"
SENTENCE = "sentence"
MARKDOWN = "markdown"
PAGE_LEVEL = "page_level"
CODE = "code"
@dataclass
class RAGChunker:
"""RAG chunking strategies with multiple splitting methods."""
def __init__(self, strategy: ChunkingStrategy = ChunkingStrategy.RECURSIVE,
chunk_size: int = 512, overlap: int = 64,
embedding_model=None):
self.strategy = strategy
self.chunk_size = chunk_size
self.overlap = overlap
self.embedding_model = embedding_model
def chunk(self, text: str, metadata: dict = None) -> list:
"""Chunk text using the selected strategy."""
if self.strategy == ChunkingStrategy.FIXED_SIZE:
return self._fixed_size_chunk(text)
elif self.strategy == ChunkingStrategy.RECURSIVE:
return self._recursive_chunk(text)
elif self.strategy == ChunkingStrategy.SEMANTIC:
return self._semantic_chunk(text)
elif self.strategy == ChunkingStrategy.SENTENCE:
return self._sentence_chunk(text)
elif self.strategy == ChunkingStrategy.MARKDOWN:
return self._markdown_chunk(text)
elif self.strategy == ChunkingStrategy.PAGE_LEVEL:
return self._page_level_chunk(text, metadata or {})
elif self.strategy == ChunkingStrategy.CODE:
return self._code_chunk(text)
else:
return self._recursive_chunk(text)
def _fixed_size_chunk(self, text: str) -> list:
"""Fixed-size chunking: split every N tokens."""
words = text.split()
chunks = []
start = 0
idx = 0
while start < len(words):
end = min(start + self.chunk_size, len(words))
chunks.append({
"content": " ".join(words[start:end]),
"chunk_index": idx,
"strategy": "fixed_size",
"token_count": end - start,
})
if end == len(words):
break
start = end - self.overlap
idx += 1
return chunks
def _recursive_chunk(self, text: str) -> list:
"""Recursive character splitting: try paragraph, then line, then word."""
separators = ["\n\n", "\n", ". ", " ", ""]
chunks = []
idx = 0
def split_recursive(text: str, seps: list, max_size: int):
nonlocal idx
if len(text.split()) <= max_size:
if text.strip():
chunks.append({
"content": text.strip(),
"chunk_index": idx,
"strategy": "recursive",
"token_count": len(text.split()),
})
idx += 1
return
sep = seps[0] if seps else ""
if sep == "":
# Fall back to fixed-size word split
parts = text.split()
for i in range(0, len(parts), max_size):
chunk_text = " ".join(parts[i:i + max_size])
if chunk_text.strip():
chunks.append({
"content": chunk_text.strip(),
"chunk_index": idx,
"strategy": "recursive",
"token_count": len(chunk_text.split()),
})
idx += 1
return
splits = text.split(sep)
current = ""
for part in splits:
candidate = current + sep + part if current else part
if len(candidate.split()) > max_size:
if current:
chunks.append({
"content": current.strip(),
"chunk_index": idx,
"strategy": "recursive",
"token_count": len(current.split()),
})
idx += 1
if len(part.split()) > max_size:
split_recursive(part, seps[1:], max_size)
current = part
else:
current = candidate
if current.strip():
chunks.append({
"content": current.strip(),
"chunk_index": idx,
"strategy": "recursive",
"token_count": len(current.split()),
})
idx += 1
split_recursive(text, separators, self.chunk_size)
return chunks
def _semantic_chunk(self, text: str,
similarity_threshold: float = 0.7) -> list:
"""Semantic chunking: split on topic transitions via embedding similarity."""
sentences = self._split_sentences(text)
if len(sentences) <= 1:
return [{"content": text, "chunk_index": 0,
"strategy": "semantic", "token_count": len(text.split())}]
# Embed each sentence
if not self.embedding_model:
# Fallback to recursive if no embedding model
return self._recursive_chunk(text)
import asyncio
embeddings = asyncio.get_event_loop().run_until_complete(
self.embedding_model.embed_batch(sentences))
# Calculate similarity between consecutive sentences
chunks = []
current_chunk = [sentences[0]]
idx = 0
for i in range(1, len(sentences)):
sim = self._cosine_similarity(
embeddings[i - 1], embeddings[i])
if sim < similarity_threshold:
# Topic transition — start new chunk
chunks.append({
"content": " ".join(current_chunk),
"chunk_index": idx,
"strategy": "semantic",
"token_count": len(" ".join(current_chunk).split()),
})
idx += 1
current_chunk = [sentences[i]]
else:
current_chunk.append(sentences[i])
if current_chunk:
chunks.append({
"content": " ".join(current_chunk),
"chunk_index": idx,
"strategy": "semantic",
"token_count": len(" ".join(current_chunk).split()),
})
return chunks
def _sentence_chunk(self, text: str,
sentences_per_chunk: int = 5) -> list:
"""Sentence-based chunking: group N sentences per chunk."""
sentences = self._split_sentences(text)
chunks = []
for i in range(0, len(sentences), sentences_per_chunk):
chunk_text = " ".join(sentences[i:i + sentences_per_chunk])
chunks.append({
"content": chunk_text,
"chunk_index": i // sentences_per_chunk,
"strategy": "sentence",
"token_count": len(chunk_text.split()),
})
return chunks
def _markdown_chunk(self, text: str) -> list:
"""Markdown header-aware chunking: split on H1/H2/H3."""
import re
# Split on markdown headers
header_pattern = r'^(#{1,3})\s+.+$'
lines = text.split("\n")
chunks = []
current_chunk = []
current_header = ""
idx = 0
for line in lines:
if re.match(header_pattern, line, re.MULTILINE):
# Save current chunk
if current_chunk:
content = "\n".join(current_chunk)
if len(content.split()) <= self.chunk_size:
chunks.append({
"content": content,
"chunk_index": idx,
"strategy": "markdown",
"header": current_header,
"token_count": len(content.split()),
})
idx += 1
else:
# Sub-chunk large sections
sub = self._recursive_chunk(content)
for s in sub:
s["strategy"] = "markdown"
s["chunk_index"] = idx
s["header"] = current_header
chunks.append(s)
idx += 1
current_chunk = [line]
current_header = line.strip()
else:
current_chunk.append(line)
if current_chunk:
content = "\n".join(current_chunk)
chunks.append({
"content": content,
"chunk_index": idx,
"strategy": "markdown",
"header": current_header,
"token_count": len(content.split()),
})
return chunks
def _page_level_chunk(self, text: str,
metadata: dict) -> list:
"""Page-level chunking: one chunk per page."""
pages = metadata.get("pages", [text])
chunks = []
for i, page in enumerate(pages):
if len(page.split()) > self.chunk_size:
# Sub-chunk large pages
sub = self._recursive_chunk(page)
for s in sub:
s["strategy"] = "page_level"
s["page_number"] = i + 1
chunks.append(s)
else:
chunks.append({
"content": page,
"chunk_index": i,
"strategy": "page_level",
"page_number": i + 1,
"token_count": len(page.split()),
})
return chunks
def _code_chunk(self, text: str) -> list:
"""Code-aware chunking: split on class/function boundaries."""
import re
# Split on function/class definitions
pattern = r'(?=\s*(?:def |class |function |const |export ))'
segments = re.split(pattern, text)
chunks = []
idx = 0
for seg in segments:
if not seg.strip():
continue
if len(seg.split()) > self.chunk_size:
sub = self._recursive_chunk(seg)
for s in sub:
s["strategy"] = "code"
s["chunk_index"] = idx
chunks.append(s)
idx += 1
else:
chunks.append({
"content": seg.strip(),
"chunk_index": idx,
"strategy": "code",
"token_count": len(seg.split()),
})
idx += 1
return chunks
def _split_sentences(self, text: str) -> list:
import re
sentences = re.split(r'(?<=[.!?])\s+', text)
return [s.strip() for s in sentences if s.strip()]
def _cosine_similarity(self, v1: list, v2: list) -> float:
import math
dot = sum(a * b for a, b in zip(v1, v2))
mag1 = math.sqrt(sum(a * a for a in v1))
mag2 = math.sqrt(sum(b * b for b in v2))
if mag1 == 0 or mag2 == 0:
return 0.0
return dot / (mag1 * mag2)
RAG Chunking Strategies Checklist
- [ ] Start with recursive character splitting at 512 tokens — it handles 80% of use cases
- [ ] Set overlap to 10-20% of chunk size (50-100 tokens for 512-token chunks)
- [ ] Start with zero overlap and add only if retrieval fails at chunk boundaries
- [ ] Overlap is not free — it increases total chunks and embedding costs
- [ ] For sentence-based chunking, use 1-2 sentences of overlap
- [ ] Use fixed-size chunking for uniform documents, prototypes, and chat logs
- [ ] Use recursive chunking for documentation, articles, and mixed corpora (default)
- [ ] Use semantic chunking only for high-value corpora with topic shifts
- [ ] Semantic chunking requires careful threshold tuning (cosine similarity 0.7)
- [ ] Semantic chunking incurs high computational cost (embed every sentence)
- [ ] Semantic chunking degrades badly under imperfect tuning — recursive is robust
- [ ] Use page-level chunking for PDFs and paginated documents (NVIDIA benchmark winner)
- [ ] Use markdown header splitter for documentation (MarkdownHeaderTextSplitter)
- [ ] Use code splitter for source code (RecursiveCharacterTextSplitter.from_language)
- [ ] Use sentence-based chunking for short-form content (Q&A, tweets, prose)
- [ ] Use token-based chunking for strict embedding budget control
- [ ] Use LaTeX splitter for academic papers with equations and proofs
- [ ] Use HTML splitter for web content (DOM-aligned)
- [ ] Use JSON splitter for API data and configuration files
- [ ] Format-aware splitting is the single biggest retrieval-quality improvement
- [ ] Mix strategies per document type rather than standardizing on one
- [ ] Add parent-document retrieval for Q&A over books or long documents
- [ ] Add sentence-window retrieval for retrieval precision with generation context
- [ ] Attach metadata to each chunk (doc_id, source, chunk_index, header, page)
- [ ] Track token_count per chunk for budget management
- [ ] Benchmark 2-3 strategies on your actual documents and queries
- [ ] Measure recall@k, precision@k, accuracy, and groundedness
- [ ] Measure RAGAS faithfulness (target: 0.8+) after chunking changes
- [ ] Evaluate before and after every chunking strategy change
- [ ] Consider late chunking (embed full document, then chunk embeddings)
- [ ] Consider LLM-based chunking for high-value, experimental content
- [ ] Consider proposition-based chunking for maximum precision (but high k needed)
- [ ] Recursive 512 with 5 chunks: 69% accuracy, 0.86 doc F1, 0.92 page F1
- [ ] Fixed 512 with 5 chunks: 67% accuracy, 0.85 doc F1, 0.85 groundedness
- [ ] Fixed 1024 with 3 chunks: 61% accuracy, 0.88 doc F1 (best for doc retrieval)
- [ ] Read what is RAG for architecture
- [ ] Read how RAG works for pipeline stages
- [ ] Build RAG from scratch with chunking
- [ ] Optimize chunk size after choosing strategy
- [ ] Understand embeddings for semantic chunking
- [ ] Choose embedding model for semantic chunking
- [ ] Add hybrid search after chunking
- [ ] Add reranking after chunking
- [ ] Prevent hallucinations with good chunking
- [ ] Evaluate with RAG metrics
- [ ] Test: each strategy produces chunks within expected size range
- [ ] Test: recursive splitting preserves paragraph boundaries
- [ ] Test: markdown splitting preserves header structure
- [ ] Test: code splitting preserves function/class boundaries
- [ ] Test: semantic chunking detects topic transitions correctly
- [ ] Document chosen strategy, chunk size, overlap, and benchmark results
FAQ
What is the best chunking strategy for RAG?
Recursive character splitting at 400-512 tokens with 10-20% overlap is the best default for most use cases. Vecta: "Recursive 512 achieved the highest accuracy (69%) across all seven strategies. Fixed 512 was close behind at 67%. Recursive splitting tries to break at natural boundaries — paragraph breaks, then sentence breaks, then word breaks. On academic text, this preserves logical units: a complete paragraph about a method, a full equation derivation. The generator gets chunks that make semantic sense." Firecrawl: "Start with RecursiveCharacterTextSplitter at 400-512 tokens with 10-20% overlap. Move to semantic or page-level only if metrics show you need extra performance." NeuroLink: "Start with recursive, measure, then optimize. The recursive strategy handles 80% of use cases well. Only switch to a format-specific strategy after you measure retrieval quality and find that chunk boundaries are causing problems." TeachMeIDEA: "Recursive chunking is the strong default. Most production systems benefit more from mixing strategies per document type."
How does semantic chunking work and when should you use it?
Semantic chunking detects topic transitions by monitoring similarity drops between consecutive sentences, aligning chunks with thematic coherence. Vecta: "Semantic chunking produced 17,481 chunks averaging 43 tokens across 50 papers. With k=46, the retriever samples from 46 different tiny chunks. The result: only 54% accuracy and 0.42 document F1. Semantic chunking requires careful threshold tuning, merging heuristics, and often parent-child retrieval to work well. When those are not perfectly dialed in, it degrades badly. Recursive splitting, by contrast, produced strong results with default parameters. The brittleness of semantic chunking under imperfect tuning is itself a finding." Firecrawl: "Semantic chunking can improve recall by up to 9% over simpler methods, at the cost of embedding every sentence." arxiv: "Semantic chunking detects topic transitions by monitoring similarity drops between consecutive sentences, aligning chunks with thematic coherence but incurring high computational cost and threshold selection challenges." Use semantic chunking only on high-value corpora with topic shifts, and only after measuring that simpler strategies fail.
What is recursive character splitting and why is it the default?
Recursive character splitting uses a hierarchy of separators (paragraphs, lines, spaces) to iteratively split oversized segments, preserving semantic coherence when possible. TeachMeIDEA: "Most production RAG systems built on LangChain or LlamaIndex use recursive character splitting as the default. The implementation is a depth-first tree split: each oversized chunk gets re-split with a smaller separator until everything fits." Vecta: "Why does recursive splitting edge out plain fixed-size? It tries to break at natural boundaries — paragraph breaks, then sentence breaks, then word breaks. On academic text, this preserves logical units: a complete paragraph about a method, a full equation derivation, a complete results discussion. The generator gets chunks that make semantic sense, not arbitrary windows that may cut mid-sentence." NeuroLink: "The Swiss Army knife. It tries to split at \n\n (paragraphs), then \n (lines), then space. Recursive strategy handles 80% of use cases well." LangChain ships RecursiveCharacterTextSplitter as the default splitter.
How much overlap should RAG chunks have?
Start with 10-20% overlap and add it only if you see retrieval failures at chunk boundaries. Firecrawl: "Industry best practices recommend 10-20% overlap as a starting point. For a 500-token chunk, use 50-100 tokens of overlap. A 1000-character chunk with 100-character overlap means the last 100 characters of chunk 1 appear as the first 100 characters of chunk 2. This helps preserve context across boundaries." NeuroLink: "Overlap is not free. Overlap increases the total number of chunks and embedding costs. Start with zero overlap and add it only if you see retrieval failures at chunk boundaries. For sentence-based chunking, 1-2 sentences of overlap is usually sufficient." Vecta: "Fixed 512 with 50-token overlap achieved 67% accuracy and 0.85 groundedness. Recursive 512 with 50-token overlap achieved 69% accuracy and 0.81 groundedness." Overlap tradeoff: better context preservation vs. more chunks and higher embedding costs.
How do you chunk markdown and code documents for RAG?
Use format-aware splitters for markdown and code rather than generic text splitters. TeachMeIDEA: "For Markdown, HTML, or code, use a format-aware splitter rather than the generic one. LangChain ships MarkdownHeaderTextSplitter for header-aware Markdown splitting, and RecursiveCharacterTextSplitter.from_language(Language.PYTHON) for source code with language-appropriate separators (class/function/block boundaries instead of paragraphs). Format-aware splitting is the single biggest retrieval-quality improvement most teams make after they realize their generic splitter shredded their codebase." NeuroLink: "Markdown documentation? Use markdown. If sections are short and fragmented, upgrade to semantic-markdown. Processing code files? Recursive with code separators — respects function/class boundaries." Firecrawl: "Processing code files: recursive with code separators. Respects function/class boundaries." For markdown: split on headers (H1, H2, H3) first, then fall back to recursive. For code: split on class/function boundaries, then block boundaries.
Want a self-hosted AI company brain that does all of this out of the box?
Book a demo →