Intermediate · Systems

Reranker

Visual diagram · (in preparation) · Math · (in preparation) · Worked example · 3 difficulty levels.

TL;DR. A second-stage model (often a cross-encoder) that re-orders an initial set of retrieved candidates for higher precision before they reach an LLM.

Technical Definition

A second-stage model (often a cross-encoder) that re-orders an initial set of retrieved candidates for higher precision before they reach an LLM.

How it works

Retrieval pipelines typically fetch 50–200 candidates with a fast bi-encoder, then apply a slower cross-encoder reranker that scores each (query, document) pair jointly with full attention. Rerankers like Cohere Rerank, BGE-reranker, and ColBERT dramatically improve top-k precision and are the single highest-leverage component in production RAG systems.

Related Concepts

• Embedding — A dense vector representation that captures semantic meaning, mapping discrete items like words into continuous mathematical space.
• Retrieval-Augmented Generation (RAG) — A technique that enhances LLM responses by retrieving relevant documents from an external knowledge base before generating an answer.
• Semantic Search — A search technology that understands the intent and contextual meaning of queries, rather than just matching keywords.