Hybrid search: why a vector DB alone isn't enough
Contents
A common story: someone puts embeddings into a vector DB and wires up search, then types in an error code or a product model number and gets back completely irrelevant results. Semantic search is strong at finding things that are "similar in meaning," but weak at queries that need an exact string match. Hybrid search is the idea of combining semantic search with keyword search to cover that weakness.
Background — why this term now
Embedding-based vector search converts sentences or words into numeric coordinates to find things that are "similar in meaning." But this conversion creates a problem: because it compresses (pools) the whole set of tokens into one fixed-size vector, the identity of strings that only matter when they match exactly — error codes like ERR_SSL_VERSION_OR_CIPHER_MISMATCH, or product model names like RTX-4090 — gets blurred. tianpan.co cites these kinds of identifiers, SKUs, function names, and rare proper nouns as classic cases vector search often misses.
Keyword search based on an inverted index (BM25), on the other hand, has the simple structure of finding a match only if the exact token is present — so searching for "vacation" won't surface "PTO." Hybrid search runs both searches in parallel to fill in each other's gaps, and has become an established way to raise retrieval quality in RAG (retrieval-augmented generation) pipelines.
Key data & current state
How hybrid search is implemented varies a bit from platform to platform.
| Platform | Fusion method | Notes |
|---|---|---|
LangChain EnsembleRetriever | Combines BM25Retriever and a vector retriever with weights (e.g. 0.5:0.5), then re-ranks with RRF (Reciprocal Rank Fusion) | Recommended to tune weights to your data |
| Weaviate | Native BM25 + vector search, with alpha (0–1) controlling the mix | Since v1.24, relativeScoreFusion, which normalizes and combines scores, is the default; previously rankedFusion, which uses only rank, was the default (Weaviate official docs) |
| Pinecone | Treats sparse and dense as one vector, scaling directly at query time by multiplying the dense value by alpha and the sparse value by (1-alpha) | For natural-language queries, alpha 0.75 (favoring the vector side) is the default recommendation (Pinecone official docs). The community has also reported that relevance drops when alpha is below 0.5 (Pinecone community), so it needs to be measured directly |
| Elasticsearch | RRF as default fusion + per-field weight tuning | In a 2025 case study on the Wands furniture dataset, default RRF raised NDCG about 1.3% over BM25 alone, and staged tuning — weighting exact matches 100x and partial matches 10x — raised it to 7.5% (tianpan.co) |
Some reports say adding reranking makes the improvement even larger. According to an Analytics Vidhya piece citing an Anthropic benchmark, combining contextualized embeddings with contextualized BM25 and then running a reranker reduced the retrieval failure rate — measured on the top 20 chunks — from 5.7% to 1.9%, about a 67% drop. However, this figure was measured by Anthropic on its own data, and it's hard to say it applies as-is across every domain.
Deep dive
1) What exactly is hybrid search
Hybrid search feeds the same query into two retrievers at once. One is dense search, which finds documents "similar in meaning" using cosine similarity between embedding vectors; the other is BM25 (sparse) search, which finds documents "containing the exact words," correcting for document length and word frequency in an inverted index. After getting result lists from both, you typically combine them one of two ways to solve the problem that their score scales differ. RRF discards the original scores and uses only rank, rescoring with 1/(k + rank) (commonly k=60); the relativeScoreFusion family normalizes both scores to a 0–1 range and then takes a weighted sum.
2) A common misunderstanding — why it reads this way
A common misunderstanding is that "vector search is smarter, so it's always superior to keyword search." This comes from the impression that embedding models understand sentence meaning, but in reality the very process of compressing meaning is the weakness. A single rare identifier in a query gets its signal weakened, buried among other tokens during the averaging (pooling) process. This problem stands out more in domains with tightly controlled vocabulary, like legal, medical, or technical documents. Another misunderstanding is thinking "just bolt on hybrid and you're done" — default fusion parameters like RRF or alpha don't fit every dataset.
3) What actually matters when judging it
Deciding whether to adopt hybrid search, and how to weight it, starts with looking at actual query logs. The first criterion is what share of queries contain identifiers that need exact matches — error codes, product numbers, function names, proper nouns. If that share is low and most queries are natural language, dense search alone may be enough. Second is how controlled the domain vocabulary is. Third is whether to use default weights (alpha 0.75, RRF k=60, etc.) as-is or tune them directly with labeled query-answer pairs. According to Bruch et al. (2022), cited by tianpan.co, tuning the weighted-sum ratio with about 40 labeled pairs consistently outperformed RRF. Fourth is whether there's room to add a reranker, which can push accuracy further at the cost of added latency and compute.
4) When it helps and when it doesn't matter
In areas like log search, code search, or product catalogs, where identifiers, numbers, and proper nouns frequently show up in queries, hybrid search is close to essential. Conversely, when queries are entirely natural language with almost no identifiers — a typical Q&A chatbot, say — it makes more sense to first measure dense-only performance. Analysis: if the dataset is small enough that a full comparison finishes quickly, the complexity of running two retrievers and tuning fusion parameters may outweigh the benefit.
In practice — 5 principles for reading hybrid search correctly
- Before adopting hybrid search, analyze actual query logs to check the share of queries containing identifiers like error codes, product names, or function names. The higher that share, the more pronounced the benefit of going hybrid.
- To solve the problem that the two searches' score scales differ, RRF is a safe default. If a platform supports score normalization methods like relativeScoreFusion, it's worth testing both approaches.
- Don't just keep the default alpha or weights (0.5–0.75) — tuning directly with a few dozen labeled query-answer pairs consistently gives better results.
- Don't stop at hybrid fusion alone; adding a cross-encoder reranker on top can widen the improvement, though it comes with added latency and cost.
- If queries are mostly natural language with few identifiers, or the dataset is small, measuring dense-only performance first and filling in only where it falls short keeps complexity down.
References
Contents
Related posts

Vector DB — A purpose-built search warehouse for AI
A vector DB converts text and images into numeric coordinates (embeddings) to quickly find semantically similar items. It’s core to RAG, but only makes sense after weighing scale, latency, and search goals.

Embedding — what it means to turn sentence meaning into coordinates
Embeddings convert sentences into numeric coordinates to cluster similar meanings. They’re core to RAG search but don’t “understand” and generate answers like LLMs—crucial when debugging odd results.
MoE (Mixture of Experts): why compute doesn't grow as much as the model does
MoE separates total parameters from the active parameters used per token. Using Mixtral and DeepSeek-V3, this explains why compute doesn't scale with model size, though memory still does.

RAG — How to keep models up to date without fine-tuning
RAG retrieves relevant documents at query time and injects them into the prompt instead of retraining the model. It shines when knowledge changes often or citations are required, but it does not eliminate hallucinations.