"RAG Core Study (24/26) — Agentic RAG: Planning, Tool Use, Verification"

As questions grow more complex, RAG starts to look less like one retrieval call and more like a small problem-solving loop.

Agentic RAG extends ordinary retrieval by adding planning, tool use, intermediate control, and verification. It does not assume one retrieval pass is enough. Instead, it treats complex questions as tasks that may require decomposition, multiple evidence sources, and explicit checking before the final answer.

0. Prerequisites

  • Part 23 Graph RAG
  • Part 22 Corrective RAG
  • Part 19 Query Routing

1. Learning Objectives

  1. Explain why Agentic RAG emerged.
  2. Distinguish planning, tool use, and verification in the retrieval pipeline.
  3. Compare standard RAG with agentic RAG.
  4. Recognise when agentic behaviour is excessive rather than helpful.

2. ํ•ต์‹ฌ ์š”์•ฝ

Agentic RAG is useful when a question cannot be answered well through one direct retrieval pass. It may decompose the question, call different retrieval tools, gather evidence in stages, and verify the answer before returning it. This is powerful for complex comparison, procedural, and relationship-heavy tasks. But it also raises latency, cost, and observability demands. Production systems should usually reserve agentic retrieval for the subset of questions that truly need it.

3. Intuition — Some Questions Need a Retrieval Plan

Question: “Why was the external-sharing exception procedure revised, and how did that change legal review?”

This question may require the system to:

  1. find the revision record
  2. identify the reason for revision
  3. find the legal review policy
  4. compare before and after implications

That is not one search. It is a sequence of evidence-gathering steps.

4. Definitions — Core Agentic Terms

Term Definition
Planning breaking the task into smaller retrieval or reasoning steps
Tool Use calling retrievers, databases, graph queries, or web search tools as needed
Verification checking whether the assembled answer matches the evidence
Reflection revising the next step after inspecting intermediate results

5. Mechanism — How Agentic RAG Extends Standard RAG

Standard RAG:

question -> retrieve -> answer

Agentic RAG:

question -> plan -> retrieve/tool -> inspect -> retrieve again if needed -> verify -> answer

The important shift is not just more retrieval. It is retrieval under explicit control.

6. Walkthrough — A Minimal Agentic Loop

6.1 Decomposing the task

subtasks = [
    "Find the latest revision record",
    "Extract the stated reason for the change",
    "Find the legal-review process impact",
]

6.2 Tool or retriever calls

for task in subtasks:
    route = choose_route(task)
    docs = retrieve(task, route)
    state.append(docs)

6.3 Verification step

if not evidence_is_consistent(state):
    state.append(retrieve("find more supporting evidence", fallback_route))

Self-explanation: What kind of question benefits most from agentic retrieval rather than a single retrieval pass?

7. Variants and Use Cases

7.1 Planner-retriever pattern

What changes
A planning step decides the retrieval subtasks before evidence collection begins.

Why it matters
Complex questions often hide multiple dependent retrieval needs.

What it enables
The system can gather evidence in stages rather than hoping one query retrieves everything.

Limit and next step
Bad planning can send the whole pipeline in the wrong direction.

7.2 Self-RAG or reflection-heavy pattern

The system checks whether the current evidence is sufficient and requests more retrieval if not.

7.3 Tool-augmented RAG

Instead of documents alone, the system may call SQL, graph queries, or external search to complement retrieval.

8. Limits and Failure Modes

8.1 Planning errors are high-leverage failures

If the decomposition is wrong, every later retrieval step can still be well executed yet fundamentally misdirected.

8.2 Cost and latency increase quickly

Multi-step retrieval and verification are expensive compared with ordinary RAG.

8.3 Observability becomes mandatory

Without traces and run records, it is extremely hard to understand where agentic retrieval went wrong.

8.4 Next step — Richer retrieval raises operational and security stakes

Once many tools and routes are in play, permissions, source controls, and re-index policies become even more important. That is Part 25.

8.5 Common Pitfalls

# Pitfall Symptom Fast Check
1 using agentic RAG for every query excess cost and latency restrict it to complex cases
2 weak planning stage wrong evidence path inspect decomposition outputs
3 no per-step trace poor debuggability log each retrieval step
4 no verification step fluent unsupported answers add evidence consistency checks
5 uncontrolled tool scope security risk constrain tool permissions explicitly

9. Self-check — Answer Before Looking

Q1. What is the main difference between standard RAG and Agentic RAG?

Answer Agentic RAG treats retrieval as a controlled multi-step workflow rather than one retrieval pass.
Why Complex tasks often need decomposition, multiple evidence paths, and verification.

Q2. What kinds of questions benefit most from Agentic RAG?

Answer Multi-step, comparison-heavy, procedural, and relationship-heavy questions.
Why They often require evidence gathered across several retrieval moves.

Q3. Why is verification especially important in Agentic RAG?

Answer Because multi-step retrieval can accumulate subtle reasoning and evidence errors.
Why Verification checks whether the assembled answer is still grounded.

Cheat Sheet — One-page Summary

Definitions - Planning: task decomposition for retrieval - Tool Use: calling different evidence sources - Verification: checking the final grounding

Minimal code

for task in subtasks:
    docs = retrieve(task, choose_route(task))

When to use what | Situation | Retrieval style | |---|---| | simple FAQ | standard RAG | | complex multi-step task | agentic RAG | | high-risk answer | stronger verification |

References

Primary sources

  • Asai, A. et al. Self-RAG. 2023.

Supporting notes

  • User notes, chapter 21 Agentic RAG

Bridge to the Next Part

Once retrieval becomes more adaptive and tool-rich, operational safety becomes central. Part 25 covers security, permissions, and re-indexing operations.

๋Œ“๊ธ€

๋Œ“๊ธ€ ์“ฐ๊ธฐ