Applied AI / Vector Databases / Module 09 · Hybrid Search and Metadata Filtering
Module 09

Hybrid Search and Metadata Filtering

Retrieval & Knowledge Vector Databases

Module 9 — Hybrid Search and Metadata Filtering

Pure vector search has blind spots. The systems that win in production combine it with keyword search and metadata filters. This module is where “good demo” becomes “trustworthy product.”

Why pure vector search isn’t enough

Embeddings capture meaning, which is great — but they can fumble the things keyword search nails:

  • Exact terms, IDs, codes: “error E1042”, “model SKU-9981”, “section 4.3(b)”. Semantically these look like noise; a keyword match finds them instantly.
  • Rare proper nouns & jargon: names, drug names, part numbers the embedding model never learned well.
  • Negation & precise wording: “not refundable” vs “refundable” can embed similarly.
  • Out-of-domain queries: if the embedding model doesn’t know your jargon, vector recall drops.

Meanwhile keyword search has the opposite blind spot: it misses synonyms and paraphrases (“car” vs “automobile”, “how do I cancel” vs “terminate my subscription”). The fix is to use both — that’s hybrid search. They cover each other’s weaknesses.

Keyword search in one minute: BM25

The classic keyword algorithm is BM25. Intuition: a document matches a query better if it contains the query’s words often (term frequency), those words are rare across the whole corpus (so they’re informative — that’s IDF, inverse document frequency), and the document isn’t artificially long (length normalization). It produces a relevance score per document using exact (or stemmed) word matches. It’s decades old, extremely fast, and still excellent for precise terms. This is also called sparse retrieval (the “vector” is mostly zeros, one slot per vocabulary word), vs. dense retrieval (embeddings).

Hybrid search: combining dense + sparse

Run both searches, then merge their result lists into one ranking. Two common merge strategies:

1. Reciprocal Rank Fusion (RRF) — simple and robust

Forget the raw scores (they’re on different scales and hard to compare). Use only the rank each result got in each list:

RRF_score(doc) = Σ over each search method  1 / (k + rank_in_that_method)
                 (k is a small constant, commonly 60)

A document ranked #1 by vectors and #3 by BM25 gets a high combined score. Items that both methods like float to the top. RRF is popular because it needs no score calibration and “just works.” Most databases (Qdrant, Weaviate, Elasticsearch, Milvus) offer it built-in.

2. Weighted score fusion (convex combination)

Normalize each method’s scores to 0–1, then blend:

final = α × vector_score + (1 − α) × keyword_score      # α tunes the balance

More control (you can favor semantics or keywords), but you must normalize scores and tune α on real data. Use when you want precise control; use RRF when you want robust defaults.

Tiny RRF implementation

def rrf(result_lists, k=60, top_n=10):
    scores = {}
    for results in result_lists:              # each is an ordered list of doc ids
        for rank, doc_id in enumerate(results):
            scores[doc_id] = scores.get(doc_id, 0) + 1.0 / (k + rank + 1)
    return sorted(scores, key=scores.get, reverse=True)[:top_n]

# vector_ids and bm25_ids are the ranked id lists from each search
final = rrf([vector_ids, bm25_ids])

Hybrid search is one of the highest-ROI upgrades in any RAG/search system. If you do nothing else fancy, do this.

Metadata filtering: searching a subset

Real queries are almost never “search everything.” They’re “search this user’s docs,” “only items in stock,” “articles after 2024,” “English only.” You attach metadata (a.k.a. payload) to each vector — tenant_id, date, category, language, permissions — and filter on it.

# Conceptual: only this tenant's recent, published docs
results = db.search(
    query_vector=qv, limit=10,
    filter={"tenant_id": "acme", "status": "published", "date": {"$gte": "2025-01-01"}}
)

This sounds trivial but is one of the hardest engineering problems in vector databases. Here’s why.

Why filtering is hard: it fights the ANN index

ANN indexes (HNSW graph, IVF cells) are built over the whole dataset to make search fast. A filter says “ignore most of that data.” Three strategies, each with a failure mode:

  1. Pre-filtering — find all matching items first, then search among them.

    • ✅ Correct results, no surprises.
    • ❌ If the filter is very selective (say 0.1% of data matches), the ANN index can’t help — you’re nearly brute-forcing the survivors. And building a filtered sub-index per query is expensive.

  2. Post-filtering — do a normal ANN search, then throw away results that don’t match the filter.

    • ✅ Fast; uses the index fully.
    • ❌ The catastrophe case: you ask for top-10, the index returns its top-10 by similarity, but none match your filter → you return zero results even though matches exist deeper in the list. You can over-fetch (ask for top-500 then filter) but you never know how many is enough.

  3. Filtered / in-search filtering (the modern answer) — teach the ANN index to check the filter during traversal, only walking to nodes/cells that satisfy it. HNSW variants prune neighbors that fail the filter as they search; IVF skips non-matching entries within cells. Qdrant’s “filterable HNSW,” Weaviate, and Milvus do versions of this.

    • ✅ Best balance of speed and correctness.
    • ❌ Complex to implement; can degrade graph connectivity if filters are extreme.

Expert insight: how well a database handles highly selective filters is one of the biggest real-world differentiators between products. Always benchmark filtered queries, not just raw vector search — many systems look great until you add a 1%-selective WHERE.

Multi-tenancy: filtering’s most important use

If you serve many customers from one database, every query must be scoped to the right tenant — both for relevance and for security (one missing filter = showing customer A’s data to customer B). Common approaches: a tenant_id filter on every query, namespaces/partitions/collections per tenant (stronger isolation, sometimes separate indexes), or partitioned indexes. Treat tenant filtering as a security boundary, not just a feature.

Putting it together: the production retrieval recipe

A strong real-world retrieval stage often looks like:

1. Apply metadata filter (tenant, permissions, recency)        ← correctness + security
2. Run dense (vector) + sparse (BM25) search in parallel        ← coverage
3. Fuse with RRF → a candidate set of ~50–100                   ← robust merge
4. Re-rank the candidates with a cross-encoder                  ← precision (Module 12)
5. Return top 5–10 to the LLM / user                            ← final

This single pipeline encapsulates most of “what experts do differently.” Each stage fixes a specific weakness of the others.

Hands-on: hybrid in Qdrant (sketch)

from qdrant_client import QdrantClient
from qdrant_client import models

client = QdrantClient(":memory:")
# Qdrant supports dense + sparse vectors and server-side fusion (RRF)
results = client.query_points(
    collection_name="kb",
    prefetch=[
        models.Prefetch(query=dense_qvec, using="dense", limit=50),
        models.Prefetch(query=sparse_qvec, using="bm25", limit=50),
    ],
    query=models.FusionQuery(fusion=models.Fusion.RRF),
    query_filter=models.Filter(must=[
        models.FieldCondition(key="tenant_id", match=models.MatchValue(value="acme"))
    ]),
    limit=10,
)

The exact API varies by database, but the shape — prefetch dense + sparse, fuse, filter, limit — is the modern standard.

Gotchas

  • Skipping hybrid because “embeddings are smarter.” They miss exact terms, IDs, and rare names. Hybrid is cheap insurance and usually a clear win.
  • Post-filtering and getting empty results. Over-fetch or use in-search filtering; don’t naively filter a top-10.
  • Not benchmarking selective filters. A database that’s fast on raw search can crawl on a 1%-selective filter. Test the real query shape.
  • Forgetting filters are security. In multi-tenant systems, a dropped tenant_id filter is a data leak. Enforce it centrally, not per-call.
  • Blindly trusting fused scores as probabilities. RRF/weighted scores rank well but aren’t calibrated relevance. Evaluate empirically (Module 11).
  • Indexing metadata you never filter on (wasted memory) or failing to index the fields you do filter on (slow filters). Match your indexing to your query patterns.

Check yourself

  1. Give one query type pure vector search handles badly and one keyword search handles badly.
  2. What does Reciprocal Rank Fusion combine, and why use ranks instead of raw scores?
  3. Explain the failure mode of naive post-filtering.
  4. Why is filtering “fighting” the ANN index, and what’s the modern resolution?
  5. Why is tenant filtering a security concern, not just relevance?

Answers: (1) Vector search fumbles exact terms/IDs/codes and rare proper nouns; keyword search misses synonyms/paraphrases. (2) It combines the rank positions a document got from each method; ranks avoid the problem that different methods’ raw scores live on incompatible scales. (3) You search first then drop non-matches, so if none of the top-k satisfy the filter you return too few (or zero) results even though matches exist deeper. (4) The index is built over the whole dataset for speed, while a selective filter wants only a small subset; the modern fix is in-search/filtered traversal that checks the filter during graph/cell exploration. (5) In multi-tenant systems a missing tenant filter returns another customer’s data — a breach, not merely an irrelevant result.

➡️ Next: 10-production-and-scaling.md — running this reliably at scale.

← Module 08: Building RAG End to End