BM25 is a text-ranking algorithm that ranks search results by how well a document matches a user's query. It rewards relevant, rare keyword matches while limiting the advantage of repeated words and unusually long documents. In software search, it is a transparent lexical retrieval method often used alone or alongside vector search.
BM25
What Is BM25?
BM25 is a text-ranking algorithm that ranks search results by how well documents match a user's query. Also called Okapi BM25 or Best Matching 25, it is a sparse, bag-of-words method for full-text search that matches terms but does not understand meaning, intent, or context.
How BM25 Calculates Search Relevance
BM25 scores each candidate document. For “API timeout error,” a page with the rarer term “timeout” may rank above one with many generic uses of “API.”
- It identifies query terms in indexed documents.
- It gives rarer terms more weight through inverse document frequency, or IDF.
- It limits the benefit of repeated terms through the k1 setting.
- It adjusts for document length through the b setting, so long pages do not win simply by having more words.
- It adds term contributions into a score and returns the highest-ranked results first.
Why BM25 Is Good for Keyword Search
BM25 is an efficient, explainable baseline for software search.
- It works quickly with inverted indexes, which map terms to documents.
- It performs well for exact terms such as error codes, API names, product IDs, and configuration keys.
- Its score can be inspected and tuned to diagnose relevance problems.
- It provides useful results before a team adds semantic retrieval.
Practical Limits of BM25
Keyword matching has limits, and results also depend on text and index configuration.
- It may miss synonyms and paraphrases that use different wording.
- It does not inherently understand user intent or word order.
- Tokenization, stemming, field boosts, and corpus quality can change results.
- Its scores rank documents for one query, not universal probabilities of relevance.
BM25 vs. TF-IDF vs. Vector Search
These methods solve related but different retrieval problems.
| Method | Matching method | Best use | Trade-off |
|---|---|---|---|
| BM25 | Exact terms with repetition and length controls | Technical keywords and document search | Weak on synonyms |
| TF-IDF | Exact terms weighted by rarity | Simple baseline ranking | Usually handles repetition and length less effectively than BM25 |
| Vector search | Similarity of text embeddings | Conceptual matches and paraphrases | Less transparent and can miss exact identifiers |
BM25 is generally preferable to basic TF-IDF when length normalization and term-frequency saturation matter. Vector retrieval helps when people describe the same concept with different words.
BM25 in LLM and Hybrid Search
In an LLM application, BM25 can retrieve exact source passages before the model writes an answer. Hybrid search combines BM25 and vector search for precise terms and semantic similarity. Reciprocal rank fusion, or RRF, merges ranked lists without requiring raw scores to be comparable.
Using BM25 in an Emergent App
Emergent can help teams build searchable documentation portals, support tools, and AI apps that retrieve trustworthy source text before responding. Explore the Emergent glossary for related concepts, or learn about app design and development and app-building software when planning a search experience. Build more searchable AI apps with Emergent.
Start Building
on Emergent today
on Emergent today
Start Building
