Advanced Retrieval Strategies

Advanced Retrieval Strategies

Advanced retrieval strategies address the gap between a model’s parametric knowledge and up‑to‑date, domain‑specific information. They matter when basic vector search returns irrelevant matches, when queries are ambiguous or syntactically specific, when latency constraints force a trade‑off with recall, or when you need to combine keyword precision with semantic breadth. This page organizes those strategies by the problem you’re trying to solve, not by a taxonomy of index types.

1. When queries are sparse or semantically distant

If a user types a short, jargon‑laden query or a phrase with low lexical overlap with your source material, a pure vector search may return nothing because the embedding space has no close neighbors. The typical remedy is a hybrid search that runs a keyword query alongside the vector query and merges results.

Concrete pattern. A curl POST to a vector‑search endpoint that accepts both a free‑text query and a vector, then merges results with reciprocal rank fusion:

curl -s -X POST https://search.example.com/v1/hybrid \
  -H "Content-Type: application/json" \
  -d '{
    "query": "B737 NG fuel pump pressure drop",
    "vector": [0.02, -0.15, 0.33, -0.01, 0.12],
    "k": 100,
    "keyword": "fuel pump pressure",
    "fusion": "rrf"
  }'

If the endpoint does not support native fusion, pull the top 50 results from each sub‑query separately and intersect them in application code.

Sharp edges.

• BM25 scores can dominate when the query contains very specific terms, starving the vector component of influence.
• Filter metadata that’s fine‑grained (e.g., doc_type: manual) may prune away vector‑relevant results if the filter field isn’t indexed for the vector backend.
• Reciprocal rank fusion assumes both sub‑lists contain some overlapping relevant items; if one list is empty, the fusion collapses to the non‑empty list, which can be misread as “no results”.
• A missing vector field or a malformed JSON array typically triggers a 400 status from the search layer, but the response body often omits which field was malformed, leading to iterative guesswork.

2. When results are relevant but poorly ranked

Even when vectors find near neighbors, the ranking may not match the user’s intent. A query for “best practices for turbine blade inspection” might retrieve documents about general inspection, but the most specific checklist gets buried.

Concrete pattern. Two‑phase retrieval: first fetch a broader set (e.g., k=200) via vector search, then re‑rank the subset with a cross‑encoder or pointwise scorer.

curl -s -X POST https://search.example.com/v1/search \
  -H "Content-Type: application/json" \
  -d '{"query": "turbine blade inspection", "vector": [-0.05, 0.21, ...], "k": 200}'

# Phase 2: re‑rank top 10 with a cross‑encoder
curl -s -X POST https://rerank.example.com/v1/rerank \
  -H "Content-Type: application/json" \
  -d '{"queries": ["turbine blade inspection"], "documents": ["<doc1_text>", "<doc2_text>", ...]}'

In practice the re‑rank step is often a lightweight model inference call; the HTTP shape above is what many managed services expose.

Sharp edges.

• Re‑ranking every retrieved set adds latency proportional to the cross‑encoder’s inference time; if k=200 and the model processes one pair per 5 ms, you add a full second.
• The cross‑encoder’s training data may not cover your domain, so it can over‑score generic similarity and under‑score domain‑specific relevance.
• If the re‑rank service is down, many pipelines silently fall back to the original vector rank, which the operator may not notice until a user complaint surfaces.
• A common confusion is treating the re‑rank score as a probability; it is typically a raw margin or similarity, not a calibrated probability, and applying a threshold as if it were a probability will mis‑filter results.

3. When latency outweighs recall

Real‑time UIs often cannot afford the ms‑scale cost of a high‑probe HNSW search or an IVF scan with a high probe factor. You may need to accept a lower recall ceiling in exchange for sub‑10‑millisecond response times.

Concrete pattern. Configure the index for approximate nearest‑neighbor search with a bounded ef or probe value, and disable re‑ranking.

# Example: creating a vector index with fixed M and ef_construction
curl -s -X PUT https://search.example.com/v1/indexes/documents \
  -H "Content-Type: application/json" \
  -d '{
    "dimensions": 768,
    "metric": "cosine",
    "params": {
      "M": 16,
      "ef_construction": 40
    }
  }'

At query time, you can trade recall by adjusting the client‑side ef parameter:

curl -s -X POST https://search.example.com/v1/search \
  -H "Content-Type: application/json" \
  -d '{"vector": [0.1, ...], "k": 5, "ef": 10}'

Sharp edges.

• The ef parameter at query time only affects the recall of the single query; it does not change the index build‑time structure, so a low ef can still suffer from poor baseline recall if the index was built with a very small M.
• IVF indexes require a probe value that selects how many lists to scan. A probe of 1 gives the fastest possible query but may miss the true nearest neighbor entirely; a probe of 100 may approach exhaustive search cost.
• Many clients set ef once at startup and never revisit it, even as the data distribution shifts, leading to silently degrading relevance without any config change.
• If the vector dimension in the query does not match the index metadata, the server returns a 400 error indicating a dimension mismatch. Operators sometimes misread this as a client‑code bug when the real cause is an outdated index rebuild.

4. When you need to blend keyword and semantic signals

Some queries benefit from both exact term matching (for typos, IDs, or categorical values) and semantic expansion (for concepts without a fixed lexical label). A hybrid query lets you specify weight or containment logic between the two signals.

Concrete pattern. A single POST payload that includes both a match clause and a knn clause, with a should block to indicate that either condition can fire, followed by a filter to narrow the result set.

curl -s -X POST https://search.example.com/v1/hybrid \
  -H "Content-Type: application/json" \
  -d '{
    "query": {
      "match": {"content": "gearbox oil rating"},
      "knn": {
        "vector": [0.3, -0.02, 0.45, ...],
        "k": 30
      }
    },
    "filter": {"product_line": "737‑800"},
    "boost": {"match": 0.6, "knn": 0.4}
  }'

The boost map is optional; if omitted, the backend applies a default blend (often equal weighting or a vendor‑specific heuristic).

Sharp edges.

• The boost map is frequently interpreted as a log‑odds weight or a simple percentage, depending on the backend; mixing the two interpretations in the same pipeline produces unexpected result distributions.
• Filters applied after the hybrid query may exclude results that satisfied one sub‑clause but not the other, effectively double‑filtering and reducing recall more than intended.
• Some backends normalize the match and knn scores to a [0,1] range before blending, while others leave them in their native scale. Without reading the docs, you won’t know which is happening, and the same payload can yield different result orders across versions.
• A silent failure mode: if the vector field in the document is sparse or missing, the knn clause may be silently ignored, and the query falls back to pure match, which the operator may attribute to a “ranking bug” when it’s really a data‑quality issue.

5. When context must be filtered or window‑limited

Retrieval is only one side of the RAG equation; the retrieved chunks must fit within the model’s context window and be semantically coherent. Strategies here involve post‑retrieval filtering, chunk size control, and metadata‑based ordering.

Concrete pattern. Retrieve more candidates than needed, then filter by a window metadata field and recency before feeding the list to the model.

# Pseudocode flow (not product‑specific)
results = vector_search(query, k=200)
filtered = [r for r in results if r.metadata["chunk_type"] == "procedure" and r.metadata["recency"] >= "30d"]
ordered = sorted(filtered, key=lambda r: r.metadata["importance_score"], reverse=True)
context = "\n".join([r.content for r in ordered[:10]])

In psql with a vector extension, you can push part of this logic into the query itself:

psql -U dev -d ai -c "
SELECT id, content, metadata->>'importance' AS importance
FROM documents
WHERE metadata->>'chunk_type' = 'procedure'
  AND (metadata->>'recency')::int >= 30
ORDER BY embedding <=> '[0.1, -0.2, ...]' LIMIT 10;
"

Sharp edges.

• JSONB metadata operations are case‑sensitive and will return no rows if the key casing differs from what was stored, a failure mode that produces an empty result set without an error message.
• The <=> operator respects the index’s distance metric; if you filter by metadata in the same WHERE clause, PostgreSQL may scan the whole table and apply the filter post‑hoc, defeating the purpose of the vector index and causing query latency to spike.
• Recency metadata often decays over time; if the ingestion pipeline tags recency at insert time but never updates it, older documents remain “fresh” in filters, leading to stale context being sent to the model.
• When the filtered set returns fewer than the requested k, some pipelines silently pad with the last retrieved chunk or with empty strings, which can cause the model to attend to padding tokens and produce nonsensical output. This is hard to debug because the model’s response appears coherent on the surface.

This page was rewritten on 17 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.