RAG vs fine-tuning: look it up or learn it in

A language model answers from its weights, the numbers it learned in training. There are two common ways to give it knowledge it lacks. Retrieval-augmented generation (RAG) looks the knowledge up for each question. Fine-tuning, more training on new examples, builds it into the weights.

1. 1
   RAG does its work at question time. Deployment is when the system goes into use. Before it, documents are cut into chunks, short passages, and stored in an index, a store that can find the chunks closest in meaning to a question. When a question arrives, the system retrieves the closest chunks, 3 by default in the rag explainer, and adds them to the prompt, the text the model reads. The weights never change. Editing a document changes the next answer once it is re-indexed, and each answer can cite the chunks it used.
2. 2
   Fine-tuning does its work before deployment. It trains the model further on examples of the behavior you want, such as questions paired with ideal answers, and that training updates the weights. At question time the prompt holds only the question, and everything else comes from the weights. Changing what the model learned takes another training run, and an answer has no document to point to.
3. 3
   RAG fits facts that are new or change often, such as prices, policies or this week's support tickets, and answers that must cite a source. Fine-tuning fits style, format and narrow skills: a house tone, a strict output format, or one specialized task such as sorting tickets by topic. It is often a weak way to add facts. Gekhman et al. (2024) found that models learn new facts slowly through fine-tuning and make up more answers once they do.