Skip to content

Retrieval-Augmented Generation (RAG)

Retrieval-Augmented Generation is a pattern where relevant external information is retrieved and supplied to a generative model when producing a response.

Retrieval-Augmented Generation, or RAG, is a pattern where relevant external information is retrieved and provided to a generative AI model when it produces a response.

It allows an application to ground responses in information that exists outside the model's original training.

How It Works

A simplified RAG workflow looks like:

User question
     ↓
Search knowledge base
     ↓
Retrieve relevant content
     ↓
Add content to model context
     ↓
Generate response

OpenAI's current retrieval tools support semantic search over external information and can combine retrieved results with models to synthesize responses. OpenAI retrieval guide

What RAG Is Useful For

RAG can be useful when a system needs information from:

  • Internal documentation
  • Current product data
  • Policies
  • Research
  • Help articles
  • Website content
  • Technical manuals
  • Organization-specific knowledge

The information can remain in an external knowledge source instead of being permanently embedded into model training.

RAG vs. Fine-Tuning

RAG and Fine-Tuning solve different problems.

RAG

Give the model relevant information at runtime.

**Fine-tuning**

Train a model further to change or reinforce its behaviour.

If the problem is:

The model doesn't know our current policy.

RAG may be appropriate.

If the problem is:

The model consistently formats or classifies this task incorrectly.

Fine-tuning may be more relevant.

OpenAI's current model-selection guidance similarly distinguishes retrieval as a tool for improving knowledge/accuracy from fine-tuning as a method for improving consistency and behaviour. OpenAI model-selection guidance

RAG Does Not Guarantee Accuracy

RAG can reduce some kinds of unsupported generation.

It does not eliminate Hallucinations.

Failures can still occur when:

  • Wrong content is retrieved
  • Relevant content is missed
  • Source content is incorrect
  • Instructions are ambiguous
  • Model misinterprets the evidence
  • Conflicting sources are supplied

The retrieval system itself therefore needs AI Evaluation.

Source Citations

For knowledge applications, it can be useful to preserve information about where retrieved content came from.

That lets the final interface provide links or citations back to original material.

This makes Human-in-the-Loop verification substantially easier.

Common Problems

Common problems include:

  • Uploading files and assuming RAG is complete
  • Poor chunking
  • No content lifecycle
  • Duplicate or contradictory documents
  • No permissions
  • Retrieval quality never evaluated
  • Model answer evaluated but source retrieval ignored
  • Old information remains indexed

How to Improve

Treat RAG as a complete information system.

Evaluate:

  1. Source quality
  2. Document processing
  3. Chunking
  4. Embeddings
  5. Retrieval
  6. Ranking
  7. Context construction
  8. Generated answer
  9. Citation/source behaviour
  10. Content updates