"RAG Core Study (18/26) — Query Rewrite & Expansion: HyDE, Step-back, Multi-query"
Users do not naturally write questions in the format retrieval models prefer. Query rewrite exists to bridge that gap without changing intent.
User questions are often short, ambiguous, and full of omitted context. Retrieval models, on the other hand, often perform better when the query is explicit, expanded, or reformatted to match the model’s training style. Part 18 introduces HyDE, Step-back prompting, Multi-query expansion, and the broader idea of model-aware query formatting.
0. Prerequisites
- Part 17 query classification
- Part 12 Hybrid Search
- Part 3 ingestion design and model-aware schema
1. Learning Objectives
- Explain why rewrite is different from answer generation.
- Distinguish HyDE, Step-back, and Multi-query strategies.
- Understand what model-aware query formatting means.
- Know when rewrite can actually make retrieval worse.
2. ํต์ฌ ์์ฝ
Query Rewrite takes the user’s question and turns it into a more retrieval-friendly form while preserving intent. HyDE writes a hypothetical answer document first and retrieves using that richer text. Step-back asks a more abstract question to recover broader conceptual context. Multi-query generates several paraphrased search queries and merges the results. Model-aware formatting adds another layer: some embedding models work better when the query resembles the format they were trained on. Rewrite helps when wording mismatch is the problem. It hurts when it changes the question’s real meaning.
3. Intuition — Why the Original Query Can Miss Good Context
Query: “What is the FX basis?”
The documents may use:
- “currency conversion principle”
- “conversion rate policy”
- “FX translation rule”
The original user wording may not match the wording that appears in the documents. Rewrite expands the search surface without changing the underlying intention.
4. Definitions — Core Rewrite Patterns
| Pattern | Definition |
|---|---|
| HyDE | Generate a hypothetical answer document, then retrieve with that text |
| Step-back | Rewrite a narrow question into a more abstract, general one |
| Multi-query | Generate several paraphrases or related search forms |
| Query Expansion | Add synonyms, aliases, abbreviations, or related terms |
| Model-aware Formatting | Phrase the query in the style an embedding model handles best |
5. Mechanism — Four Main Reasons to Rewrite
Rewrite is usually needed because:
- the user query is too short
- the document wording differs from the question wording
- the embedding model prefers a different query shape
- the question is too narrow and misses the broader concept that anchors the answer
So rewrite is not decoration. It is a repair step for likely retrieval mismatch.
6. Walkthrough — HyDE, Step-back, and Multi-query
6.1 HyDE
def hyde_query(query, llm):
prompt = f"Write a short document that would answer: {query}"
hypothetical_doc = llm.generate(prompt)
return hypothetical_doc
search_text = hyde_query(query, llm)
hits = dense_retriever.search(search_text, k=20)
HyDE gives Dense retrieval a richer semantic object than the raw question alone.
6.2 Step-back
step_back_query = "What is the policy for currency conversion in financial reporting?"
hits = retriever.search(step_back_query, k=20)
This helps when the original question is too narrow and the relevant documents use broader framing.
6.3 Multi-query
queries = [
query,
"What is the currency conversion policy?",
"FX translation rule for reporting",
]
all_hits = [retriever.search(q, k=10) for q in queries]
The retrieved lists can then be fused or reranked.
Self-explanation: Why is rewrite not the same as replacing the user’s question with a different one?
7. Variants and Use Cases
7.1 HyDE
What changes
The system retrieves with a generated hypothetical answer text.
Why it matters
Dense retrieval can benefit when the search text looks more like an answer-bearing document than a short user query.
What it enables
Stronger semantic retrieval for underspecified queries.
Limit and next step
If the hypothetical document drifts too far from intent, recall may worsen.
7.2 Step-back
This is useful when the answer lives in more general explanatory or policy documents than the user’s surface wording suggests.
7.3 Multi-query
When terminology varies across sources, multiple formulations can improve recall substantially.
7.4 Model-aware query formatting
Some embedding models are more effective when queries are full questions, use explicit prefixes, or reflect the format seen in training.
8. Limits and Failure Modes
8.1 Rewrite can distort intent
If the original question is time-sensitive or precision-sensitive, a more abstract rewrite may move away from the real answer.
8.2 Rewrite adds latency and cost
If you call an LLM for every query, rewrite becomes a real operational stage rather than a harmless tweak.
8.3 Multi-query increases noise as well as recall
More queries mean more candidate documents, which can force rerankers to work harder.
8.4 Next step — Once queries are rewritten, they still need to be routed correctly
Better wording is not enough if the system searches the wrong collection or wrong retriever path. That is Part 19.
8.5 Common Pitfalls
| # | Pitfall | Symptom | Fast Check |
|---|---|---|---|
| 1 | rewriting every query | unnecessary latency | gate rewrite by query type |
| 2 | no intent check | rewrites drift semantically | compare original and rewritten meaning |
| 3 | too many paraphrases | reranker overload | measure recall vs candidate noise |
| 4 | ignoring embedding model format | weak retrieval lift | inspect model card guidance |
| 5 | abstracting time-sensitive questions | stale or wrong answers | preserve recency constraints |
9. Self-check — Answer Before Looking
Q1. What is the main purpose of query rewrite?
Answer To make retrieval see the user’s intent more clearly without changing that intent.
Why Document wording and user wording often differ.
Q2. What is distinctive about HyDE?
Answer It retrieves using a generated hypothetical answer document.
Why Dense retrieval can work better with richer semantic text than a short raw query.
Q3. When is Step-back useful?
Answer When the original query is too narrow and the answer lives in broader conceptual material.
Why Abstracting the query can expose the correct explanatory documents.
Q4. Why can rewrite hurt retrieval?
Answer Because a rewrite may drift away from the user’s actual need.
Why Rewrite is itself a model decision that can be wrong.
Cheat Sheet — One-page Summary
Definitions - HyDE: hypothetical answer document retrieval - Step-back: more abstract reformulation - Multi-query: several parallel paraphrased queries
Minimal code
queries = [query, rewrite_1, rewrite_2]
hits = [retriever.search(q, k=10) for q in queries]
When to use what | Situation | Rewrite pattern | |---|---| | short ambiguous question | HyDE | | missing broader context | Step-back | | terminology variation | Multi-query | | embedding-model format mismatch | model-aware formatting |
References
Primary sources
- Gao, L. et al. Precise Zero-Shot Dense Retrieval without Relevance Labels. 2022.
Supporting notes
- User notes, chapter 15 query rewrite
- User notes, section 35-3 query formatting
Bridge to the Next Part
Once the query is improved, the next decision is where to send it: which collection, which retriever, and which fallback path? Part 19 covers query routing.
๋๊ธ
๋๊ธ ์ฐ๊ธฐ