Retrieval-Augmented Generation (RAG)
Retrieval-augmented generation (RAG) is a pattern that retrieves selected external material at request time and supplies it as context to a generative model.
RAG can make answers more grounded and updateable, but it does not guarantee that retrieval, interpretation or the answer is correct.
At a glance:
- Meaning: retrieve relevant material, place it in model context, then generate a response.
- Why it matters: teams can use governed sources without putting every changing fact into model training.
- Used by: knowledge search, service, operations, research and internal-assistance teams.
- Commonly confused with: search, fine-tuning, a database, long-term memory and citations.
What RAG means and how it works
The original 2020 paper by Lewis and coauthors described RAG as combining a pretrained parametric model with non-parametric memory accessed through retrieval, then evaluated particular research tasks and datasets. Modern business implementations use the label more broadly, but the central separation remains: retrieval selects external context; generation produces the answer. Read the original paper.
RAG is not a source of truth by itself. Source ownership, document boundaries, indexing, permissions, freshness, retrieval, context assembly, answer policy and verification are separate components.
User request + identity/purpose:
→ form a retrieval query
→ search permitted indexed content
→ rank and filter candidate passages
→ assemble context with source metadata
→ generate a bounded answer
→ validate, cite, abstain or hand off
The retrieval unit might be a passage, record or structured field. Its metadata should retain source, owner, version, permission and freshness. A high similarity score means “close under this retrieval method,” not “true” or “authorized.”
Example and boundaries with related terms
A fictional support assistant answers an employee's expense-policy question. Identity permits access to the global policy and the employee's region addendum. Retrieval returns the current global section, an expired regional PDF and a current regional exception. The pipeline removes the expired document by effective date, applies the current exception, and cites both surviving sources. When the employee asks whether a receipt exception is legally required, the assistant abstains and routes the question to the policy owner. No compliance or service outcome is claimed.
Concept | Meaning | Use when | Do not confuse with |
|---|---|---|---|
| Search | Returns ranked documents or items | Reader should inspect results | Generated synthesis |
| RAG | Retrieves context for a generated answer | Synthesis across permitted sources is useful | Guaranteed factuality |
| Fine-tuning | Adjusts model behavior/weights from training examples | Stable behavior or task adaptation is needed | Current knowledge store |
| Tool/API lookup | Calls a system for structured current state | Exact transactional data is required | Semantic document retrieval |
| Memory | Preserves selected state across interactions | Prior context is permitted and useful | Authoritative knowledge |
When RAG matters and what can go wrong
RAG is useful when knowledge changes more often than model training, answers need traceable sources, or access differs by user. It is a poor default when exact calculation, transactional state, a deterministic policy decision or a direct database/API response is required.
Failures can occur at every stage: missing documents, stale versions, wrong identity, bad chunk boundaries, misleading embeddings, weak ranking, context overload, instruction-bearing documents, unsupported synthesis and decorative citations. NIST's Generative AI Profile discusses broader risks such as confabulation and the need for evaluation and monitoring; it does not certify RAG as a mitigation. Review the NIST profile.
Evaluate retrieval recall on a labeled set, source-version correctness, permission violations, answer support, abstention, citation entailment, latency and accepted handoff. A fluent answer is not the pass condition.