# RAG vs Fine-Tuning: When to Use Which

- Date: 2026-08-22
- Tags: ai, llm, rag, fine-tuning

## The Short Answer

- **RAG**: When your knowledge changes frequently, or when you need citations and traceability.
- **Fine-tuning**: When you need a specific output style, tone, or format that prompting alone can't reliably achieve.

## Retrieval-Augmented Generation

RAG keeps the base model frozen and augments it with a retrieval step — fetching relevant chunks from a vector store before generation. The model sees the retrieved context and generates a grounded response.

**Pros:**
- Updatable knowledge without retraining
- Transparent sourcing (you can cite exactly which chunk influenced the answer)
- Works with smaller base models

**Cons:**
- Retrieval quality caps generation quality — garbage in, garbage out
- Latency overhead from embedding + search

## Fine-Tuning

Fine-tuning continues training on task-specific data, baking knowledge or behavior directly into the weights.

**Pros:**
- Faster inference (no retrieval step)
- Better at learning style, format, and domain-specific reasoning patterns

**Cons:**
- Expensive to update as knowledge evolves
- Risk of catastrophic forgetting
- Requires high-quality labeled data

## My Rule of Thumb

Start with RAG. If you're fighting the model's default style rather than its knowledge, then consider fine-tuning — but only after you've exhausted prompt engineering.

---

Back to the full index: /agent