RAG Reranking: Cross-Encoders, Cohere, BGE, and FlashRank Compared

TL;DR — RAG reranking uses cross-encoders to re-score retrieved candidates, improving precision by 10-20%. MachineLearningMastery: "A good retriever gets you close. A good reranker gets you to the right answer. In 2026, adding a reranker is essential." LocalAIMaster: "Bi-encoder vs cross-encoder vs ColBERT trade-offs. Models: BGE-Reranker-v2-m3, Cohere Rerank 3, Jina Reranker v2, mxbai-rerank, Voyage rerank-2." Inventiple: "256-token chunks + Cohere reranking beats 1024-token without. Reranking picks the best ones from candidates." AIWorkflowLab: "FlashRank: p50=76ms. Cohere Rerank: p50=230ms. Profile latency — TTFT budget must include retrieve + rerank." DevTo: "Cross-encoder processes query + chunk together, capturing cross-attention. Slower but significantly more accurate." Learn more with hybrid search, what is RAG, build from scratch, and chunk size.

MachineLearningMastery frames the value: "Reranking is a simple yet powerful way to improve a RAG system. A good retriever gets you close. A good reranker gets you to the right answer. In 2026, adding a reranker is essential. Benchmarks like MS MARCO and BEIR consistently show that adding a reranker improves nDCG by 10-20%."

LocalAIMaster covers the landscape: "The complete reranker guide for RAG. Bi-encoder vs cross-encoder vs ColBERT trade-offs. Models compared: BGE-Reranker-v2-m3, Cohere Rerank 3, Jina Reranker v2, mxbai-rerank, Voyage rerank-2. Setup, latency benchmarks, fine-tuning."

Reranking Architecture

Reranker Models Comparison

Bi-Encoder vs Cross-Encoder vs ColBERT

Implementation

import asyncio
import time
from dataclasses import dataclass
from typing import Optional
from enum import Enum

class RerankerType(Enum):
    COHERE = "cohere"
    BGE = "bge"
    FLASHRANK = "flashrank"
    JINA = "jina"
    COLBERT = "colbert"
    MXBAI = "mxbai"

@dataclass
class RAGReranker:
    """RAG reranking with multiple cross-encoder models."""

    def __init__(self, reranker_type: RerankerType,
                 model_name: str = None,
                 api_key: str = None,
                 device: str = "cpu"):
        self.reranker_type = reranker_type
        self.model_name = model_name
        self.api_key = api_key
        self.device = device
        self._model = None

    async def rerank(self, query: str, candidates: list,
                     top_k: int = 5,
                     over_fetch: int = 3) -> list:
        """Rerank candidates using cross-encoder.

        Args:
            query: User query string
            candidates: List of retrieved candidates from Stage 1
            top_k: Number of final results to return
            over_fetch: Multiplier for candidates to rerank

        Returns:
            Top-k candidates sorted by rerank score
        """
        start = time.time()

        # Over-fetch: rerank more than top_k, then select best
        rerank_count = min(len(candidates), top_k * over_fetch)
        to_rerank = candidates[:rerank_count]

        # Create (query, document) pairs
        pairs = [
            (query, c["payload"]["content"])
            for c in to_rerank
        ]

        # Score with cross-encoder
        scores = await self._score_pairs(pairs)

        # Attach rerank scores
        for candidate, score in zip(to_rerank, scores):
            candidate["rerank_score"] = float(score)

        # Sort by rerank score descending
        ranked = sorted(
            to_rerank,
            key=lambda x: x["rerank_score"],
            reverse=True,
        )

        rerank_time = (time.time() - start) * 1000

        # Return top_k with timing
        results = ranked[:top_k]
        for r in results:
            r["rerank_time_ms"] = round(rerank_time, 1)

        return results

    async def _score_pairs(self, pairs: list) -> list:
        """Score query-document pairs with selected reranker."""
        if self.reranker_type == RerankerType.COHERE:
            return await self._score_cohere(pairs)
        elif self.reranker_type == RerankerType.BGE:
            return await self._score_bge(pairs)
        elif self.reranker_type == RerankerType.FLASHRANK:
            return await self._score_flashrank(pairs)
        elif self.reranker_type == RerankerType.JINA:
            return await self._score_jina(pairs)
        elif self.reranker_type == RerankerType.COLBERT:
            return await self._score_colbert(pairs)
        else:
            return [0.0] * len(pairs)

    async def _score_cohere(self, pairs: list) -> list:
        """Score with Cohere Rerank 3 API."""
        # In production: cohere.Client(api_key).rerank(...)
        # Simulated scores for demonstration
        return [self._simulate_score(q, d) for q, d in pairs]

    async def _score_bge(self, pairs: list) -> list:
        """Score with BGE-Reranker-v2-m3 (self-hosted)."""
        # In production: FlagModel('BAAI/bge-reranker-v2-m3')
        return [self._simulate_score(q, d) for q, d in pairs]

    async def _score_flashrank(self, pairs: list) -> list:
        """Score with FlashRank (ultra-fast, lightweight)."""
        # In production: from flashrank import Ranker
        return [self._simulate_score(q, d) for q, d in pairs]

    async def _score_jina(self, pairs: list) -> list:
        """Score with Jina Reranker v2."""
        # In production: jina.Client(api_key).rerank(...)
        return [self._simulate_score(q, d) for q, d in pairs]

    async def _score_colbert(self, pairs: list) -> list:
        """Score with ColBERTv2 (late interaction)."""
        # In production: ColBERTv2 model
        return [self._simulate_score(q, d) for q, d in pairs]

    def _simulate_score(self, query: str, doc: str) -> float:
        """Simulate cross-encoder scoring (deterministic)."""
        # In production: use actual model
        # Here: simple word overlap as proxy
        query_words = set(query.lower().split())
        doc_words = set(doc.lower().split())
        overlap = len(query_words & doc_words)
        total = len(query_words | doc_words)
        if total == 0:
            return 0.0
        return overlap / total

    def get_latency_profile(self) -> dict:
        """Get expected latency profile for this reranker."""
        profiles = {
            RerankerType.COHERE: {"p50_ms": 230, "p95_ms": 410,
                                   "note": "API network round-trip"},
            RerankerType.BGE: {"p50_ms": 50, "p95_ms": 120,
                               "note": "Self-hosted GPU"},
            RerankerType.FLASHRANK: {"p50_ms": 76, "p95_ms": 145,
                                      "note": "Ultra-fast, CPU"},
            RerankerType.JINA: {"p50_ms": 100, "p95_ms": 200,
                                "note": "API or self-hosted"},
            RerankerType.COLBERT: {"p50_ms": 40, "p95_ms": 90,
                                    "note": "Token-level interaction"},
            RerankerType.MXBAI: {"p50_ms": 60, "p95_ms": 130,
                                  "note": "Self-hosted GPU"},
        }
        return profiles.get(self.reranker_type, {"p50_ms": 100,
                                                  "p95_ms": 200})

RAG Reranking Checklist

• [ ] Add a reranker to your RAG pipeline — in 2026, it is essential
• [ ] A good retriever gets you close; a good reranker gets you to the right answer
• [ ] Reranking improves nDCG by 10-20% over retrieval-only
• [ ] 256-token chunks + reranker beats 1024-token chunks without reranker
• [ ] Cross-encoders process query + document together (full cross-attention)
• [ ] Bi-encoders embed query and document separately (fast, less accurate)
• [ ] ColBERT: late interaction — separate encoding + token-level interaction
• [ ] ColBERT is a middle ground between bi-encoder speed and cross-encoder accuracy
• [ ] Over-fetch candidates (3x top_k) from retrieval for reranking
• [ ] Create (query, document) pairs for each candidate
• [ ] Score each pair with cross-encoder model
• [ ] Sort by rerank score descending
• [ ] Return top_k final results after reranking
• [ ] Cohere Rerank 3: API, multilingual, 230ms p50, $0.12/1M tokens
• [ ] BGE-Reranker-v2-m3: self-hosted, multilingual, ~50ms p50 (GPU), free
• [ ] FlashRank: ultra-fast, lightweight, 76ms p50, free, English only
• [ ] Jina Reranker v2: API + self-hosted, 89 languages, ~100ms p50
• [ ] mxbai-rerank: open-source, competitive, ~60ms p50 (GPU)
• [ ] Voyage rerank-2: API, enterprise, ~200ms p50
• [ ] ColBERTv2: late interaction, ~40ms p50, token-level matching
• [ ] RankLLM: listwise reranker using LLM, ~500ms, highest reasoning quality
• [ ] Profile latency — TTFT budget must include retrieve + rerank time
• [ ] FlashRank: p50=76ms, p95=145ms (lightweight)
• [ ] Cohere Rerank API: p50=230ms, p95=410ms (network round-trip)
• [ ] Self-hosted cross-encoder on GPU: tens of milliseconds
• [ ] Choose reranker based on latency budget, deployment model, and language
• [ ] For multilingual: use Cohere Rerank 3 or BGE-Reranker-v2-m3
• [ ] For self-hosted: use BGE-Reranker-v2-m3 or FlashRank
• [ ] For fastest: use FlashRank (76ms p50)
• [ ] For best API quality: use Cohere Rerank 3
• [ ] For balanced: use Jina Reranker v2
• [ ] Always benchmark reranker on your domain data before deploying
• [ ] Rerankers trained on MS MARCO may struggle with domain-specific content
• [ ] Rerankers trained on English may struggle with non-English content
• [ ] If reranking worsens results: check language, domain, candidates, compatibility
• [ ] Fine-tune reranker on your domain if off-the-shelf models underperform
• [ ] Compare retrieval metrics with and without reranking
• [ ] Measure nDCG, recall@k, precision@k before and after reranking
• [ ] Reranking is Stage 2 in the 4-stage pipeline: retrieve → fuse → rerank → generate
• [ ] Reranking works best after hybrid search (BM25 + dense + RRF)
• [ ] Do not rerank only top_k — over-fetch and rerank 3x top_k
• [ ] Consider listwise rerankers (RankLLM) for complex multi-criteria ranking
• [ ] Consider ColBERT for token-level matching without full cross-encoder cost
• [ ] Read hybrid search for Stage 1
• [ ] Read what is RAG for architecture
• [ ] Build RAG from scratch with reranking
• [ ] Optimize chunk size — smaller + rerank beats larger
• [ ] Apply chunking strategies before retrieval
• [ ] Choose embedding model for bi-encoder
• [ ] Prevent hallucinations with precise retrieval
• [ ] Evaluate with RAG metrics
• [ ] Test: reranking improves nDCG over retrieval-only
• [ ] Test: 256-token + reranker beats 1024-token without
• [ ] Test: over-fetching 3x top_k produces better results than reranking only top_k
• [ ] Test: cross-encoder scores differ from bi-encoder similarity scores
• [ ] Test: reranker does not worsen results on your domain data
• [ ] Document reranker model, latency profile, and benchmark results

FAQ

What is reranking in RAG and why is it important?

Reranking re-scores retrieved candidates using a cross-encoder that processes query and document together, capturing cross-attention relationships that bi-encoders miss. MachineLearningMastery: "A good retriever gets you close. A good reranker gets you to the right answer. In 2026, adding a reranker is essential." LocalAIMaster: "The complete reranker guide for RAG. Bi-encoder vs cross-encoder vs ColBERT trade-offs. Models compared: BGE-Reranker-v2-m3, Cohere Rerank 3, Jina Reranker v2, mxbai-rerank, Voyage rerank-2." Inventiple: "Retrieval returns candidates; reranking picks the best ones. A cross-encoder reranker reads both the query and each passage together and scores relevance more accurately than embedding-based similarity. 256-token chunks with Cohere reranking consistently outperformed 1024-token chunks without reranking." Reranking improves precision by 10-20% and allows smaller chunks (256 tokens + reranker beats 1024 without).

What is the difference between bi-encoder and cross-encoder?

Bi-encoders embed query and document separately (fast but less accurate), while cross-encoders process them together (slower but more accurate). LocalAIMaster: "Bi-encoder: query and document are encoded independently, then similarity is computed. Fast but misses cross-attention. Cross-encoder: query and document are concatenated and processed together, capturing full cross-attention. Slower but significantly more accurate. ColBERT: late interaction — encodes separately but computes token-level interactions at search time. Middle ground between bi-encoder speed and cross-encoder accuracy." Medium: "Cross-encoders process query + chunk together as a single input, capturing cross-attention relationships. It is slower but significantly more accurate." Bi-encoder: O(n) embeddings, O(1) search. Cross-encoder: O(n) pairwise scoring at query time. ColBERT: O(n) embeddings, O(n×m) token-level interaction.

Which reranker should you use for production RAG?

Choose based on latency budget, deployment model, and language requirements. MachineLearningMastery: Top 5 rerankers: (1) Cohere Rerank 3 — API, multilingual, easy setup. (2) BGE-Reranker-v2-m3 — open-source, self-hosted, multilingual. (3) FlashRank — ultra-fast, lightweight, free. (4) Jina Reranker v2 — API + self-hosted, multilingual. (5) mxbai-rerank — open-source, competitive quality. LocalAIMaster: "Profile latency — TTFT budget should explicitly include retrieve + rerank time." AIWorkflowLab: "Hybrid + FlashRank rerank top 50: p50=76ms, p95=145ms. Hybrid + Cohere Rerank API top 50: p50=230ms, p95=410ms." Decision: (1) Fastest/free: FlashRank (76ms p50). (2) Best API: Cohere Rerank 3 (230ms p50, multilingual). (3) Best self-hosted: BGE-Reranker-v2-m3 (free, multilingual). (4) Balanced: Jina Reranker v2.

How do you implement reranking in a RAG pipeline?

Implement reranking by over-fetching candidates from retrieval, creating (query, document) pairs, scoring with cross-encoder, and selecting top_k. DevTo: "Stage 2: Cross-Encoder Reranking — a cross-encoder processes query + chunk together as a single input, capturing cross-attention relationships. Pairs = [(query, chunk_content) for chunk in candidates]. Scores = reranker.predict(pairs). Sort by rerank_score descending." Inventiple: "Over-fetch candidates (3x top_k) for reranking. Create (query, document) pairs. Score with cross-encoder. Sort by score. Return top_k." Glukhov: "Reranking with Embedding Models, Qwen3 Embedding + Qwen3 Reranker on Ollama." Steps: (1) Retrieve 3x top_k candidates from hybrid search. (2) Create (query, chunk_content) pairs for each candidate. (3) Score each pair with cross-encoder model. (4) Sort candidates by rerank score descending. (5) Return top_k final results.

Can reranking worsen RAG results?

Yes, reranking can worsen results if the reranker is trained on different data than your domain or language. Reddit: "Most popular rerankers are trained on English MS MARCO data and struggle with domain-specific or multilingual content. If your docs are not English, this explains a lot." MachineLearningMastery: "If a newer model does not significantly outperform BGE, the added cost or latency may not be justified." Mitigation: (1) Always benchmark reranker on your domain data before deploying. (2) Use multilingual rerankers (BGE-Reranker-v2-m3, Cohere Rerank 3) for non-English content. (3) Fine-tune reranker on your domain if off-the-shelf models underperform. (4) Compare retrieval metrics with and without reranking. (5) If reranking worsens results, check: wrong language model, domain mismatch, insufficient candidates, or reranker not compatible with your embedding model.

Want a self-hosted AI company brain that does all of this out of the box?
Book a demo →