A cross-encoder is a neural ranking model that reads two inputs, such as a search query and a document, together and assigns them a relevance score. It is commonly used as a reranker because joint processing captures subtle relationships that separate text embeddings can miss. Its improved precision comes with a speed and cost trade-off, so it usually evaluates a small set of already retrieved results.
Cross-Encoder
What Is a Cross-Encoder?
A cross-encoder is a neural ranking model that reads two inputs together, usually a search query and a candidate document, then gives the pair a relevance score. Unlike an embedding model, it does not create a reusable vector for each input. Its job is to judge how well one specific item answers or matches another.
Cross-encoders are commonly used as rerankers in search, question answering, duplicate detection, and retrieval-augmented generation, or RAG. They can notice details such as negation, word order, and the relationship between terms in both inputs.
How a Cross-Encoder Reranker Works
A cross-encoder usually improves an existing retrieval system rather than searching an entire corpus by itself.
- Receive a user query, such as “reset a forgotten administrator password.”
- Use keyword, vector, or hybrid search to retrieve a bounded candidate set.
- Pair the query with each candidate passage or document.
- Process each pair jointly, allowing words in one input to attend to words in the other.
- Assign each pair a relevance score.
- Sort candidates by score and return the highest-ranked results.
The score ranks alternatives for that model and task. It is not automatically a probability unless the model has been specifically calibrated.
Cross-Encoder vs. Bi-Encoder
These models solve different stages of search. A bi-encoder finds candidates quickly, while a cross-encoder makes the final ordering more precise.
| Feature | Cross-Encoder | Bi-Encoder |
|---|---|---|
| Input | Query and document together | Each input separately |
| Output | One pairwise relevance score | Reusable embeddings |
| Indexing | Cannot precompute pair scores | Document embeddings can be indexed |
| Best role | Reranking a small candidate set | Fast retrieval across large collections |
Benefits of Cross-Encoders
Joint processing can improve relevance when the meaning depends on a precise relationship between the query and text.
- Captures fine-grained wording, context, and word order.
- Handles ambiguous queries more effectively than similarity alone.
- Improves the passages supplied to an answer-generating model.
- Supports semantic search, question answering, and duplicate detection.
Practical Limits of Cross-Encoders
Greater precision has a compute and latency cost.
- Each query-candidate pair requires a separate model run.
- Reranking too many results can make search slow and expensive.
- Long documents may be truncated, hiding useful evidence.
- Quality depends on representative training data and relevance testing.
- Rerank a fixed, small candidate set rather than an entire corpus.
Using a Cross-Encoder in RAG and AI Search
In RAG, vector search may retrieve several plausible support articles, but a cross-encoder can promote the passage that directly answers the question. Choose a model that fits the domain, test rankings offline against labeled queries, and monitor production results for latency, incorrect citations, and sensitive-data exposure. Teams planning an AI product can start with the Emergent glossary, explore app design and development, and compare AI agent builders. Emergent can help build and ship AI search or RAG apps with a reranking layer. Build precise AI search apps with Emergent.
Frequently Asked Questions
Your Questions, Answered
This will automatically populate, don't change
Don't change this element unless you know what you are doing
What is a cross-encoder reranker?
A cross-encoder reranker scores a query and each retrieved result together, then reorders the results by predicted relevance. It usually runs after keyword or vector search has produced a smaller candidate set.
What does a cross-encoder do?
It evaluates the relationship between two inputs, such as a question and a passage, and returns a score for that specific pair. It is designed for comparison, not for producing standalone searchable embeddings.
What is the difference between a cross-encoder and a bi-encoder?
A bi-encoder encodes queries and documents separately, making it fast enough for large-scale retrieval. A cross-encoder processes both together, which is slower but can make more detailed relevance judgments.
What is reranking in a RAG system?
Reranking is the step that reorders passages retrieved for a RAG system before they are sent to a language model. It helps provide more relevant context and can reduce answers based on loosely related passages.
When should you use a cross-encoder?
Use a cross-encoder when relevance quality matters and you can first narrow results to a manageable number. It is especially useful for support search, knowledge bases, legal or technical document retrieval, and RAG applications.
Start Building
on Emergent today
on Emergent today
Start Building
