FollowAgents AI AGENTS DIRECTORY
Home / AI Agent Glossary / Agentic RAG
Ecosystem & Emerging Terms

Agentic RAG

Also called: agentic retrieval-augmented generation

Agentic RAG is retrieval-augmented generation restructured as an agent loop: instead of retrieving once and generating, the model plans a query, retrieves, judges whether the results are sufficient, reformulates or switches sources, and repeats until it can answer.

PlanRetrieveCheckAnswer
Agentic RAG turns retrieval into a loop the model controls — deciding what to fetch, judging whether it's enough, and re-querying — instead of a single up-front lookup.

Classic retrieval-augmented generation is a fixed two-step pipeline: embed the user's question, pull the top matching chunks from a vector store, and stuff them into the prompt for the model to answer from. It works well when the answer sits in one place and the question is phrased close to the source. It struggles with multi-hop questions, ambiguous phrasing, or cases where the first search misses.

Agentic RAG makes retrieval a decision the model drives, inside a loop. The model plans what to look for, issues a retrieval call as a tool (function-tool-calling), reads the results, and judges: is this enough to answer, or do I need to search again with a better query, or query a different source? It might decompose a question into sub-questions, retrieve for each, and combine. The structure is the agent-loop / react-prompting pattern applied to search.

By 2026 this is a mainstream production pattern rather than a research curiosity. The tradeoff is cost and latency — several model calls and retrieval round-trips instead of one — plus a need for guardrails so the loop terminates. It's used when questions are genuinely multi-step or the knowledge is spread across sources, and plain RAG is still the right choice when they aren't.

How it works

The loop typically covers: planning (what information is needed, broken into sub-queries if useful); routing (which index, database, API, or web search is the right source for each part); retrieval via a tool call; evaluation (are the results relevant and sufficient? the model or a separate check decides); and reformulation (rewrite the query, try another source, or narrow/broaden) before looping again. It exits when the model judges it has enough, or a step budget is hit. Compared with plain RAG, the extra machinery is the model's control over query formulation, source choice, and stopping — none of which are fixed in advance.

Example

Question: "Did the framework our payments service depends on change its license after the version we pinned?" Plain RAG retrieves chunks about the framework's license and likely answers from whatever it finds. Agentic RAG plans two sub-questions — which version is pinned, and what the license history is — retrieves the lockfile for the first, retrieves the framework's changelog and license file for the second, notices the changelog reference is ambiguous, re-queries for the specific version range, and only then answers, citing both sources.

How it differs

Agentic RAG vs. classic RAG: classic RAG does one embedding search and passes the top-k chunks to the model in a fixed pipeline. Agentic RAG lets the model iterate — rewrite queries, choose among sources, decompose the question, and decide when to stop — at the cost of more model calls, higher latency, and the need to bound the loop.

Common misconceptions

Often assumed: Agentic RAG is just classic RAG with a bigger top-k.
Actually: Retrieving more chunks is still one fixed step; agentic RAG adds a control loop where the model reformulates queries, picks sources, and decides whether to retrieve again.
Often assumed: Agentic RAG is always better, so it should replace plain RAG.
Actually: It costs several times the calls and latency; for direct, single-hop questions plain RAG is faster and just as accurate.

FAQ

What is agentic RAG?
Retrieval-augmented generation done as an agent loop: the model plans a query, retrieves via a tool call, judges whether the results are enough, reformulates or switches sources, and repeats until it can answer — instead of a single retrieve-then-generate step.
How is agentic RAG different from normal RAG?
Normal RAG is a fixed pipeline — one embedding search, top-k chunks into the prompt. Agentic RAG lets the model iterate: rewrite the query, choose sources, break the question down, and decide when to stop.
When should you use agentic RAG?
For multi-hop or ambiguous questions, or when the answer is spread across several sources. For direct single-hop lookups, plain RAG is cheaper and fast enough.

Last checked: 2026-08-30

Related terms

ReAct

ReAct is a prompting technique that has a language model explicitly interleave written reasoning ('Thought') with actions and their results ('Observation'), rather than jumping straight from a question to an action.

Agent Loop

The agent loop is the repeating perceive-plan-act-observe cycle by which an AI agent turns a goal into a sequence of actions, checking each result before deciding what to do next.

Function Calling / Tool Calling

Function calling (also called tool calling) is a model capability where the model can output a structured request to invoke a specific function with specific arguments, instead of only producing free-form text.