Hybrid Retrieval: BM25 + Vector Search for Production RAG

TL;DR — Hybrid retrieval combines BM25 keyword search with vector semantic search to cover each method's blind spots. BM25 matches exact terms, product codes, and proper nouns that vector search misses. Vector search matches paraphrases and semantic intent that BM25 misses. Results are fused using Reciprocal Rank Fusion (RRF), which combines rank positions rather than scores, avoiding the score-incompatibility problem. The default RRF constant k=60 is a reliable starting point. For production RAG, the pattern is: BM25 + vector search in parallel, RRF fusion, optional cross-encoder reranking for the top candidates. Elasticsearch and OpenSearch support native hybrid search with RRF. pgvector can approximate it with PostgreSQL full-text search. Hybrid retrieval is the baseline for enterprise RAG — pure vector search is no longer sufficient.

Pure vector search has a problem: it can't find exact matches. If you search for "ISO 27001" and your document says "ISO/IEC 27001:2022," vector search might rank it lower than a document that discusses "information security standards" semantically. BM25 would find the exact match immediately.

Pure BM25 has the opposite problem: it can't find paraphrases. If you search for "how to handle data breaches" and your document says "incident response procedures for security compromises," BM25 finds no keyword overlap. Vector search would match the semantic intent.

Hybrid retrieval runs both methods in parallel and fuses the results. This covers each method's blind spots and produces more accurate retrieval than either method alone.

BM25 vs Vector Search: What Each Misses

How Does Hybrid Retrieval Work?

Step 1: Parallel Retrieval

Both BM25 and vector search run simultaneously against the same corpus:

class HybridRetriever:
    """Combines BM25 and vector search with RRF fusion."""

    def __init__(self, bm25_store, vector_store, rrf_k: int = 60):
        self.bm25_store = bm25_store
        self.vector_store = vector_store
        self.rrf_k = rrf_k  # RRF constant, 60 is standard

    def retrieve(self, query: str, top_k: int = 10) -> list:
        # Run both retrievers in parallel
        bm25_results = self.bm25_store.search(query, top_k=top_k * 3)
        vector_results = self.vector_store.search(query, top_k=top_k * 3)

        # Fuse with Reciprocal Rank Fusion
        fused = self._rrf_fuse(bm25_results, vector_results)

        # Return top_k results
        return fused[:top_k]

    def _rrf_fuse(self, list1: list, list2: list) -> list:
        """Reciprocal Rank Fusion: combine by rank, not score."""
        scores = {}

        for rank, result in enumerate(list1):
            doc_id = result["id"]
            scores[doc_id] = scores.get(doc_id, 0) + 1 / (self.rrf_k + rank + 1)

        for rank, result in enumerate(list2):
            doc_id = result["id"]
            scores[doc_id] = scores.get(doc_id, 0) + 1 / (self.rrf_k + rank + 1)

        # Sort by fused score
        sorted_ids = sorted(scores, key=scores.get, reverse=True)

        # Return with fused scores
        all_results = {r["id"]: r for r in list1 + list2}
        return [
            {**all_results[doc_id], "fused_score": scores[doc_id]}
            for doc_id in sorted_ids
        ]

Step 2: Reciprocal Rank Fusion (RRF)

RRF combines rankings rather than scores. This is critical because BM25 scores and cosine similarity scores are on completely different scales — you cannot average them directly.

The RRF formula:

RRF_score(d) = Σ 1 / (k + rank_i(d))

Where:
- d is a document
- rank_i(d) is the document's rank in result list i (1-indexed)
- k is a constant (default: 60) that dampens the influence of high ranks

Why k=60? The original Cormack et al. 2009 paper found k=60 to be a robust default. Lower k (30-40) favors top-1 precision. Higher k (80-100) favors top-10 recall. For most enterprise RAG use cases, k=60 is the right starting point.

Step 3: Optional Cross-Encoder Reranking

After RRF fusion, a cross-encoder can rerank the top candidates for additional precision:

class RerankingHybridRetriever(HybridRetriever):
    """Hybrid retrieval with cross-encoder reranking."""

    def __init__(self, bm25_store, vector_store, reranker, rrf_k=60):
        super().__init__(bm25_store, vector_store, rrf_k)
        self.reranker = reranker

    def retrieve(self, query: str, top_k: int = 10) -> list:
        # Get top candidates from hybrid retrieval
        candidates = super().retrieve(query, top_k=top_k * 5)

        # Rerank with cross-encoder
        reranked = self.reranker.rerank(
            query=query,
            documents=[c["content"] for c in candidates],
            top_k=top_k
        )

        return reranked

The pattern: retrieve top-50 with hybrid search, rerank top-10 with a cross-encoder. This adds 50-200ms latency but improves precision significantly.

Which Databases Support Hybrid Search?

pgvector Hybrid Search

For self-hosted deployments already using PostgreSQL, pgvector can approximate hybrid search:

-- Hybrid search in PostgreSQL: BM25 (tsvector) + pgvector
WITH bm25_results AS (
    SELECT id, content, 
           ts_rank_cd(tsv, query) AS bm25_score,
           ROW_NUMBER() OVER (ORDER BY ts_rank_cd(tsv, query) DESC) AS bm25_rank
    FROM documents, plainto_tsquery('english', $1) query
    WHERE tsv @@ query
    LIMIT 50
),
vector_results AS (
    SELECT id, content,
           1 - (embedding <=> $2) AS vector_score,
           ROW_NUMBER() OVER (ORDER BY embedding <=> $2) AS vector_rank
    FROM documents
    ORDER BY embedding <=> $2
    LIMIT 50
)
-- RRF fusion
SELECT id, content,
       COALESCE(1.0 / (60 + bm25_rank), 0) + 
       COALESCE(1.0 / (60 + vector_rank), 0) AS rrf_score
FROM bm25_results FULL OUTER JOIN vector_results USING (id)
ORDER BY rrf_score DESC
LIMIT 10;

This requires maintaining a tsvector column for BM25 alongside the embedding column for vector search. The RRF fusion is implemented in SQL. For production use, consider OpenSearch which provides native hybrid search with built-in RRF.

When to Use Hybrid Retrieval

Rule of thumb: If your corpus contains proper nouns, product codes, IDs, technical terminology, or exact phrases that users will search for, use hybrid retrieval. If your corpus is purely conversational or conceptual, vector search alone may suffice.

Performance Considerations

Hybrid Retrieval Implementation Checklist

• [ ] Choose database: OpenSearch (self-hosted), Elasticsearch, or pgvector (PostgreSQL)
• [ ] Create BM25 full-text index alongside vector embedding index
• [ ] Configure RRF fusion with k=60 as starting point
• [ ] Run both retrievers in parallel (not sequentially)
• [ ] Set top_k for each retriever to 3-5x the final desired result count
• [ ] Apply RRF fusion to combine ranked lists
• [ ] Optionally add cross-encoder reranking for top candidates
• [ ] Enforce document-level ACLs in both retrieval pipelines
• [ ] Benchmark hybrid vs vector-only on your actual query distribution
• [ ] Measure precision@k, recall@k, and MRR (mean reciprocal rank)
• [ ] Tune RRF k parameter if needed (lower k for precision, higher for recall)
• [ ] Monitor latency: hybrid should be under 100ms without reranking
• [ ] For pgvector: maintain tsvector column with trigger updates
• [ ] For OpenSearch: use native hybrid query with RRF join mode
• [ ] Consider stopping hallucinations with grounded retrieval
• [ ] Evaluate RAG vs knowledge graphs for multi-hop queries
• [ ] Test with real enterprise queries containing exact terms and paraphrases

FAQ

What is hybrid retrieval in RAG?

Hybrid retrieval combines sparse keyword search (BM25) with dense vector semantic search. BM25 matches exact terms and phrases. Vector search matches meaning and intent. By running both in parallel and fusing results, hybrid retrieval captures queries that either method alone would miss. The fusion is typically done with Reciprocal Rank Fusion (RRF), which combines rankings rather than scores, avoiding the score-incompatibility problem.

Why use BM25 alongside vector search?

Vector search fails on exact keyword matches, rare terms, product codes, and names. BM25 fails on paraphrases, synonyms, and semantic intent. BM25 excels at: exact term matching, proper nouns, IDs and codes, and queries with specific terminology. Vector search excels at: semantic similarity, paraphrased queries, and conceptual matching. Using both covers each method's blind spots.

What is Reciprocal Rank Fusion (RRF)?

Reciprocal Rank Fusion (RRF) is a rank-based fusion algorithm that combines results from multiple retrieval methods. Instead of trying to normalize incompatible scores (BM25 scores and cosine similarity scores are on different scales), RRF uses only the rank positions. The formula is: RRF_score = sum(1 / (k + rank_i)) for each result list i, where k is typically 60. RRF outperforms naive score averaging because it avoids the score-incompatibility problem.

Which databases support hybrid search natively?

Elasticsearch and OpenSearch support native hybrid search with RRF. Pinecone supports sparse-dense co-located indexes. Weaviate offers native hybrid search (alpha parameter controls the mix). Qdrant supports sparse vectors. pgvector can approximate hybrid search using PostgreSQL full-text search alongside vector similarity, but fusion logic must be implemented manually. For self-hosted deployments, OpenSearch is the most complete option.

Should you add cross-encoder reranking to hybrid search?

Cross-encoder reranking adds a final accuracy boost by re-scoring the top candidates from hybrid retrieval with a more expensive but more accurate model. The pattern is: retrieve top-50 with hybrid search, rerank top-10 with a cross-encoder. This improves precision but adds latency (50-200ms). Add reranking when your latency budget allows and when the accuracy improvement justifies the cost. Start with RRF-only hybrid search, measure, then add reranking if needed.

Want a self-hosted AI company brain that does all of this out of the box?
Book a demo →