Retrieval-Augmented Generation (RAG)

What is retrieval-augmented generation?

Retrieval-Augmented Generation, or RAG, is a technique that retrieves relevant information from an external source and supplies it to a language model before the model generates an answer. It helps the model use private, specialized, or current information that may not be reliably available in its trained parameters.

RAG does not change the model itself. It changes the [context](/glossary/context/) used for a particular inference.

How RAG works

A common RAG pipeline follows these steps:

1. Convert the user’s question into a search query or embedding.
2. Retrieve relevant document chunks from an index or database.
3. Optionally filter or rerank the candidates.
4. Place the strongest evidence into the model’s context.
5. Ask the model to answer using that evidence.

For example, an internal support agent may retrieve the latest warranty policy and the customer’s product guide, then use those passages to explain the correct process.

RAG versus retrieval

Retrieval is the search step that finds information. RAG is the complete pattern that combines retrieval with model generation. A search feature can retrieve documents without generating an answer, while a model can generate text without retrieval.

RAG is also not the same as agent memory. Memory stores information for later use. RAG is a way to find and inject information at runtime. An agent may use RAG to retrieve memories, documents, or both.

Key design choices

RAG quality depends on more than the language model. Builders must decide how to split documents into chunks, create embeddings, combine keyword and semantic search, apply metadata filters, rerank results, and fit evidence within the context window.

The system should preserve source details so users or downstream checks can verify important claims. Sensitive documents also require access controls during retrieval, not only after generation.

Common failure modes

RAG can retrieve irrelevant, stale, duplicated, or unauthorized content. Even with correct evidence, the model may ignore it, combine passages incorrectly, or make unsupported claims. A missing result does not prove that the information does not exist.

Reliable systems evaluate retrieval separately from answer generation. They test whether the correct evidence appears, whether ranking is useful, and whether responses remain grounded in the supplied material.

Why it matters for agents

RAG lets agents consult external knowledge while working instead of relying only on model knowledge or enormous prompts. It is especially useful for changing policies, enterprise data, technical documentation, and large knowledge collections.

RAG improves access to evidence, but it does not guarantee truth. Good agent design combines retrieval with source awareness, permissions, validation, and clear behavior when evidence is missing or conflicting.