StackLift
All terms

Glossary

Reranking

A second pass that reorders retrieved results by how well they answer the question.

Reranking is a second retrieval step in which a more precise model rescores a short list of candidate passages and puts the most relevant ones first. The first search is fast and broad, and the reranker, often a cross-encoder, reads the query and each passage together to judge the match. Only the top few passages are then passed to the language model.

The usual pattern is to retrieve a generous set of candidates, rerank them, and keep a handful. Rerankers cost more per item than a vector lookup, which is why they run only on the short list.

Reranking is one of the most common fixes when a RAG system retrieves roughly the right material but ranks it badly. It adds latency and cost, so it should be tested against your own questions before you keep it.

Start in 60 seconds

Put this to work on your project.

Get an AI-assisted, architect-reviewed scope, cost range, and timeline. Free, no login.

The complete guide

Reranking: the complete guide

Updated November 18, 2026 ยท Published by , Founder, StackLift AI. Written with AI assistance; how we make content.

Short answer

Reranking is a second search step that reorders a short list of retrieved results by how well each one answers the question. A fast first search finds many candidates, then a more precise model reads the query and each candidate together and puts the best ones first. It improves what a language model sees in a RAG system.

What is reranking?

Reranking means sorting search results a second time with a better judge. The first search is built for speed: it scans a large collection and returns a few dozen plausible matches. The reranker is built for precision: it looks closely at each match and decides which ones really answer the question.

Think of hiring. A resume screen picks a pool of candidates quickly. Interviews then rank that small pool carefully. You would not interview everyone, and you would not hire from the screen alone. Reranking is the interview stage for search.

It matters most in RAG, where only a handful of passages fit into the model's prompt. Which handful you pick decides whether the answer is right.

Why does reranking matter?

First-stage retrieval, whether keyword or vector based, is good at recall and mediocre at ordering. The right passage often appears at position seven, not position one. If you pass only the top three to the model, the answer is lost. If you pass all ten, you add noise and cost and may confuse the model.

  • Better answers: the model gets the most relevant passages and less distraction.
  • Lower cost per answer: fewer, better passages mean a shorter prompt.
  • Less sensitivity to chunk size and phrasing: the reranker corrects some retrieval misses.
  • Better search results for people too, not only models, in site and document search.
  • A tunable quality lever that does not need retraining your main model.

How does reranking work, step by step?

  1. A user asks a question or runs a search.
  2. The first stage retrieves a generous candidate set using semantic search, keywords or hybrid search.
  3. Each candidate is paired with the query and sent to the reranker.
  4. The reranker assigns every pair a relevance score.
  5. Candidates are sorted by score, and low scorers can be dropped by a threshold.
  6. The top few are used: shown to the user, or passed to the language model as context.
  7. The system logs scores and outcomes so you can tune the cutoffs.

The key difference from the first stage is that the reranker reads the query and the passage together. A vector search compares two summaries made separately. A reranker can notice that a passage mentions the right words but answers a different question.

What types of rerankers are there?

Common reranking approaches
ApproachHow it worksStrengthTrade-off
Cross-encoderA model reads the query and passage as one input and outputs a scoreAccurate and widely usedCost grows with each candidate scored
Hosted rerank APIA vendor service returns scores for your candidatesQuick to adoptAnother vendor and data path to approve
LLM as rerankerA language model is asked to rank or score candidatesFlexible, follows custom criteriaSlower and costlier per call
Late interaction modelsCompare token-level representations of query and passageGood balance of precision and speedMore complex to host
Rule-based boostingAdjust scores by recency, source, authority or popularitySimple and transparentDoes not understand meaning

Many production systems combine them: a model-based reranker for relevance, then business rules for freshness, source trust or user permissions.

What are examples of reranking in use?

  • A customer support assistant retrieves many help articles, then reranks them so the one that matches the exact error message comes first.
  • A legal research tool finds many clauses on termination and reranks them by relevance to a specific jurisdiction or contract type.
  • An e-commerce search finds products by meaning, then reranks by fit with the query, availability and margin rules.
  • An internal knowledge assistant reranks policy documents so the current version outranks an archived one.
  • A developer search tool reranks code and documentation snippets for a specific question.
  • An AI agent reranks tool documentation or past cases before choosing its next action.

In each case the first stage is cheap and broad, and the second stage spends more effort on a small set. That is the whole idea.

When should you add reranking, and when should you skip it?

Add reranking when your tests show that the right passage is usually retrieved but not ranked near the top, or when you must pass very few passages to the model. Skip it when the collection is small, answers are easy to find, or latency and cost limits leave no room for another step.

Decision guide
SituationReranking?
Right passage appears in the top candidates but not the top fewYes, this is the classic case
Right passage is missing from candidatesNo. Fix chunking, metadata or search first
Tiny, clean document setProbably not needed
Strict response-time limitTest it; use a small reranker or fewer candidates
Mixed content types and long documentsOften yes

What tools and models are used for reranking?

You can use a hosted rerank service, an open model that you run yourself, or a general language model prompted to score results. Hosted services are the quickest to start with. Open models give you control over where data goes, which matters when passages are sensitive. Prompted language models are handy for prototypes and for custom ranking rules.

Where reranking fits in a typical stack
LayerExamples
First-stage searchPostgreSQL with pgvector, Elasticsearch, Qdrant, Weaviate. See vector database vs. pgvector
RerankerHosted rerank APIs, open cross-encoder models, or a prompted LLM such as OpenAI, Anthropic Claude or Google Gemini models
OrchestrationLangGraph or custom pipeline code
EvaluationA fixed question set with expected sources, plus logging and tracing

If you search with Elasticsearch, Qdrant or Weaviate, you can usually add a rerank step in the application layer without changing the index.

Is reranking only for RAG?

No. Reranking started in classic search and recommendation systems, and it is still used there. Any system that finds many candidates cheaply and then needs the best few benefits from the same pattern: product search, job matching, document discovery, support ticket routing and recommendation lists.

What changed with language models is the stakes. A person scanning a results page can skip a poor first hit. A model given a poor passage may build a confident answer on it. That makes ordering and thresholds more important in RAG than in a plain search box, and it is why a reranking step often pays for itself there first.

It also helps with the limits of a model's context window. Even when a model accepts long input, filling it with weak passages raises cost and can lower accuracy. A tight, well-ordered context is usually cheaper and clearer.

How do you implement reranking?

  1. Build a test set of real questions with the correct source passage for each.
  2. Measure retrieval alone: is the right passage among the candidates, and at what position?
  3. Choose a reranker and a candidate count that gives it room to help without slowing every query.
  4. Add the reranker, then measure again with the same test set.
  5. Set a score threshold so low-quality matches are dropped and the system can say it did not find an answer.
  6. Add business rules for freshness, source and permissions after relevance scoring.
  7. Monitor latency, cost and quality, and re-run the test set whenever data or models change.

Keep the reranker behind a clean interface so you can swap it. Models in this area change quickly, and a swap should be a configuration change, not a rewrite.

What mistakes should you avoid?

  • Reranking a very short candidate list, which leaves nothing to reorder.
  • Reranking too many candidates, which adds delay for little gain.
  • Using a reranker that does not handle your language or domain vocabulary.
  • Ignoring permissions: filter content by access before reranking, so restricted passages are never scored or shown.
  • Judging by a few hand-picked examples instead of a repeatable test set.
  • Treating the score as a probability. It is a relative ranking signal.

Most failures trace back to the first stage. Check recall before blaming the reranker.

Another quiet problem is a stale reranker. If your documents, terminology or user questions change over time, scores that once looked right can drift. Re-run your test set on a schedule, and when a new model arrives, compare it on your data before switching, not on someone else's benchmark.

Finally, keep a plain fallback. If the reranking service is slow or down, the system should return the first-stage order rather than fail. Quality drops a little, and users still get an answer.

How do you measure whether reranking is working?

Compare the same test questions with and without reranking. The questions to ask are simple: is the correct passage now higher in the list, does the final answer improve, and what did the extra step cost in time and money?

  • Position of the right passage before and after reranking.
  • Share of questions where the right passage reaches the model.
  • Answer quality as judged on your test set. See agent evaluation.
  • Added latency per query and added cost per query.
  • User behavior in production: reformulated questions, thumbs down, escalations to a person.

Log the scores so you can see where the cutoff should sit and when quality drifts. AI observability covers the tracing needed.

What does reranking mean for cost, timeline and how StackLift works?

Reranking is a small part of a search or RAG system in effort, and a visible part of its running cost, because each query scores several candidates. The effort goes to building the test set, tuning candidate counts and thresholds, and wiring in permissions. See the RAG implementation playbook and the AI LLM integration cost page.

StackLift scopes retrieval work as separate deliverables: ingestion, search, reranking, permissions, evaluation and monitoring. Estimates are reviewed by a senior architect, priced by deliverable complexity rather than hours, and observability is planned from sprint one. The free estimate shows cost, timeline and team in about five minutes.

Questions people ask

Is reranking the same as semantic search?

No. Semantic search finds candidates by meaning. Reranking reorders a short list of candidates more precisely. They are used together.

Do I need reranking for RAG?

Not always. It helps when the right passage is retrieved but ranked too low, or when only a few passages fit in the prompt. Test retrieval first.

What is a cross-encoder?

A model that reads the query and a passage together and outputs a relevance score. It is more accurate than comparing separate vectors, but slower, so it runs on a short list.

Can a large language model rerank results?

Yes. You can ask a model to score or order candidates. It is flexible and follows custom criteria, but it usually costs more and takes longer than a dedicated reranker.

How many results should I rerank?

Enough to give the reranker real choice, but not so many that every query slows down. Tune it on your own test set, since the right number depends on your data and latency limits.

Does reranking add latency?

Yes, a little, because it adds a step. Smaller rerankers, fewer candidates and hosting close to your data keep the delay small.

Does reranking replace hybrid search?

No. Hybrid search improves which candidates are found. Reranking improves their order. Many strong systems use both.

Is reranking worth it for a small document set?

Often not. With few, clean documents, plain retrieval already finds the answer. Add reranking when tests show that ordering is the problem.

Sources

Sources and further reading

The standards, rules, research and official documentation this page draws on.