Retrieval-Augmented Generation
Why Retrieval?
LLMs have knowledge cutoffs — retrieval brings fresh facts
A 2022 cutoff covers 4 of 8 events. Events after 2022 produce hallucinations because the model has no training data for them.
Step 1 of 6: Why Retrieval?
LLMs have knowledge cutoffs — retrieval brings fresh facts
A 2022 cutoff covers 4 of 8 events. Events after 2022 produce hallucinations because the model has no training data for them.
Step 2 of 6: Document Embedding
Split documents into chunks and embed each as a vector
200-token chunks produce 5 balanced chunks. Each is embedded into a dense vector — enough context per chunk for meaningful retrieval.
Step 3 of 6: Query & Retrieval
Find the nearest chunk embeddings to the query vector
Top-3 retrieval balances recall and noise. The 3 nearest chunks in embedding space are returned, giving the model multiple evidence sources.
Step 4 of 6: Similarity Search
Filter chunks by cosine similarity threshold
Threshold 0.50 filters to 8 chunks. Cosine similarity measures the angle between query and chunk vectors — higher means more semantically aligned.
Step 5 of 6: Augmented Prompt
Inject retrieved context into the prompt template
3 chunks using 2008 tokens. Good balance — enough retrieved evidence to ground the answer, with 2008 tokens left for a thorough response.
Step 6 of 6: Faithfulness & Hallucination
Better retrieval grounds more tokens in real documents
Moderate retrieval quality — 15 grounded tokens, 5 hallucinated. The grounding score of 75% shows room for improvement via better chunking or reranking.