Welcome Welcome to Lumen Labs. This handbook covers pay, time off, equipment and remote work. It was last updated in March 2025.
Time off Every new hire starts with a vacation allowance. In the first year it is 15 days. After three years it rises to 20 days. Up to five unused days carry over.
Sick leave Sick leave is separate from vacation. Every employee gets 10 paid sick days per year. A doctor's note is needed after three days in a row.
Archived 2022 policy Archived 2022 policy: new employees received 10 days of paid holiday per year, with no carry-over.
Public holidays The office closes on 11 public holidays. These do not count against your vacation balance.
Remote work Staff may work from home up to 3 days a week. A laptop and a monitor are provided for the home office.
Pay Salaries are paid on the last working day of each month. Overtime is paid at 1.5 times the hourly rate.
Parental leave Birth parents get 16 weeks of paid leave. Other parents get 8 weeks of paid leave.
Equipment IT gives every employee a laptop. Lost laptops must be reported to IT within 24 hours.
Expenses Travel costs are reimbursed within 30 days when receipts are submitted.
Lumen Labs and its handbook are made up. The search, scores and answers are computed live.
A real handbook300 pages
Tokens to paste this whole handbook
253
A 300-page handbook (≈500 tokens a page)
150,000
Tokens with RAG: question + 3 chunks
105
"How many vacation days do new hires get?" The answer is in the Time off section (green), but a language model has never read this handbook: it's private, and it changed in March 2025, after any model's training data was collected. Without it the model can only guess, and it will usually guess fluently. For a handbook this small you could paste all 253 tokens into every prompt. A 300-page one would be about 150,000 tokens per question, more than a 128,000-token context window holds. Retrieval-augmented generation (RAG) instead finds the few passages that matter, 105 tokens here, and puts only those in the prompt.
Pick a question and drag the handbook size. A model's knowledge is frozen at its training cutoff and never includes your private documents, so for questions about them it can only guess. Long context windows let you paste a lot, but every token is paid for on every question and models get worse at using facts buried in the middle of very long prompts. RAG (Lewis et al., 2020) keeps the documents in a searchable index and fetches only what each question needs, so the knowledge can be updated without retraining the model.
Step 2 of 6: Cutting Documents Into Chunks
Chunk size decides whether a fact keeps its context
question:
Time off
Every new hire starts with a vacation allowance. In the first year it is 15 days. After three years it rises to 20 days. Up to five unused days carry #1over. #10
Archived 2022 policy
Archived 2022 policy: new employees received 10 days of paid holiday per year, with no carry-over. #2
Words per chunk30
Chunks in the handbook
11
"allowance" and "15 days" in one chunk?
yes
Top retrieved chunk
"Every new hire starts with a vacation allowance. In the first year it is 15 days. After three years it rises to 20 days. Up to five unused days carry"
Answer from the top 3 chunks
15 days
Before search, the handbook is cut into chunks: here, windows of 30 words inside each section. The #n tag is each chunk's rank for the question. At 30 words "Every new hire starts with a vacation allowance" and "In the first year it is 15 days" land in the same chunk, so the top chunk carries both the topic and the number, and the answer comes out as 15. Too small and facts lose their context; too large and each chunk mixes several topics and costs more prompt tokens. Real systems often split on sentence or heading boundaries and let neighbouring chunks overlap.
Drag the chunk size down until "allowance" and "15 days" fall into different chunks, then watch the answer change. Each chunk is indexed and retrieved on its own, so it has to make sense on its own. Common choices are a few hundred tokens with some overlap, or splitting on paragraphs and headings. Some systems also store a short summary or the section title with each chunk, which is one way to keep the topic attached to the fact.
Step 3 of 6: Keyword vs Embedding Search
Matching words, or matching meaning
question:
search:
#1
Archived 2022 policy: new employees received 10 days of paid holiday per year, with no carry-over.
1.53
#2
Time off: Every new hire starts with a vacation allowance. In the first year it is 15 days. After three years it rises to 20 days. Up to five unused days carry over.
1.22
#3
Welcome to Lumen Labs. This handbook covers pay, time off, equipment and remote work. It was last updated in March 2025.
0.00
#4
Sick leave is separate from vacation. Every employee gets 10 paid sick days per year. A doctor's note is needed after three days in a row.
0.00
#5
Public holidays: The office closes on 11 public holidays. These do not count against your vacation balance.
0.00
Question words found in the handbook
new
Concepts the toy embedding maps to
PTO → time off · starters → new hire
Rank of the right section
#2
Keyword search (BM25) scores chunks by the question's words they contain, weighting rare words more. The handbook never uses "pto" or "starter", so only "new" can match, and the right section only reaches #2. The archived 2022 policy wins instead. Switch to embedding search.
Compare keyword and embedding search on the PTO and telecommute questions. BM25 ranks by shared words, weighting rare ones more, and it has no idea that PTO means vacation. Embedding search compares meaning vectors with cosine similarity, which handles synonyms and paraphrases. In practice each covers the other's blind spots: embeddings can miss exact product names, codes and numbers that keyword search finds, which is why many production systems run both and merge the rankings (hybrid search), often followed by a reranking model.
Step 4 of 6: The Augmented Prompt
What goes in decides what comes out
question:
search:
chunks in the prompt (top-k):
Answer using only the context. Cite the chunk number.
[1] Archived 2022 policy: new employees received 10 days of paid holiday per year, with no carry-over.
Question: How much PTO do new starters get?
Answer: 10 [1], from "Archived 2022 policy: new employees received 10 days of paid holiday per year, with no carry-over."
Prompt size
44 tokens
Answer
10
Correct? (the current handbook says 15)
no: stale source
The retrieved chunks are pasted into the prompt, numbered, above the question, with an instruction to answer only from them and cite. The reader here is simple: it answers with the context sentence closest to the question that contains a number (read together with the sentence before it). It answered 10, faithfully quoting [1], the archived 2022 policy. The answer is grounded and still wrong: retrieval handed it an outdated document. Raise k so the current policy gets in, switch to embedding search, or skip archived documents, which is what metadata filters are for. The model can only be as right as what retrieval gives it.
Try top-k = 1 with keyword search on the PTO question, then raise k or skip archived documents. The prompt template puts the retrieved chunks, numbered, above the question and asks the model to answer only from them and cite its sources. That makes answers checkable, but it can't fix a bad retrieval: an outdated or wrong document in the context leads to a grounded, cited, wrong answer. Metadata filters (date, version, permissions) and deduplication of old documents are as important as the search itself.
Step 5 of 6: Checking Every Claim
Faithful to the context is not the same as true
example answer:
Retrieved for "How many vacation days do new hires get?": [1] Time off · [2] Archived 2022 policy · [3] Sick leave
New hires get 15 vacation days in their first year.
supported by [1] Time off: "In the first year it is 15 days." (similarity 0.65)
Unused vacation days can be cashed out at the end of the year.
not supported: the closest context sentence only reaches 0.25
Claims supported by the context
1 / 2
Faithfulness score
0.50
Supported only by a stale source?
no
These are example answers a model might write (written for this demo); the checking is computed. Hallucination is a property of claims, not of words, so each answer is split into claims and each claim is checked against the retrieved chunks: it counts as supported when some context sentence shares its number and is close in the toy embedding. The first claim is supported. The second, about cashing out, appears nowhere in the context: it's a hallucination, so faithfulness is 1/2. Tools such as RAGAS compute this kind of score with an LLM as the judge instead of a fixed threshold.
Switch between the three drafts and toggle archived documents. Faithfulness asks whether every claim in an answer is backed by the retrieved context; it is checked claim by claim, because one unsupported sentence can hide among correct ones. Faithfulness is not correctness: an answer that faithfully repeats a stale source scores perfectly. Evaluation frameworks such as RAGAS split answers into claims with an LLM and ask another LLM whether each claim follows from the context.
Step 6 of 6: Measuring the Pipeline
Retrieval recall and answer accuracy, side by side
search:
top-k:
Words per chunk60
Recall@1: right section in the top 1
7 / 9
Answers correct
7 / 9
Correct answers when retrieval hit
7 / 7
To improve a RAG system, measure the two halves separately over a set of questions with known answers. Recall@k asks whether the right passage made it into the top k; answer accuracy asks whether the final answer is right. With keyword search, top-1 and 60-word chunks, recall is 7/9 and 7/9 answers are right. Every miss in the answers traces back to a miss in retrieval: the reader can't use what it never sees. A caveat: the toy embedding was given synonyms for exactly these questions, so it looks better than a real one would. Real evaluations use hundreds of questions written by people who didn't build the index.
Change the search method, top-k and chunk size and watch the two scores move. Recall@k tests the retriever: did the right passage reach the prompt? Answer accuracy tests the whole system. Measuring both tells you where to work: if recall is low, improve chunking, search or filters; if recall is high but answers are wrong, improve the prompt or the model. Real evaluation sets have hundreds of realistic questions with known sources, and the numbers here look better than reality because the toy embedding was built with these questions in mind.
What is retrieval-augmented generation (RAG)?
Retrieval-augmented generation (RAG) is a technique that looks up relevant documents for a question and adds them to a language model's prompt before it answers. The documents are split into chunks and stored as embedding vectors. At query time the question is embedded too, the closest chunks are retrieved, and the model writes its answer from that context.
RAG is how most chatbots answer questions about information they were not trained on: a company's internal documents, a product manual, recent news, or a codebase. It keeps answers current without retraining, lets the system cite its sources, and reduces made-up facts. Customer support assistants, enterprise search, coding assistants, and search engines with AI answers are all built on some form of it.
Key ideas
RAG retrieves relevant documents at query time and adds them to the prompt before the model answers.
Documents are split into chunks, and each chunk is stored as an embedding vector.
The query is embedded with the same model, and the nearest chunks by cosine similarity are retrieved.
Chunk size and top-k both trade retrieval precision against recall.
Editing the document index changes what the system knows without retraining the model.
Good retrieval reduces hallucination, but the model can still ignore or misread its context.
What is the difference between RAG and fine-tuning?
Fine-tuning changes the model's weights and is best for teaching a style, format, or behavior. RAG leaves the weights alone and supplies facts at query time, which makes the facts easy to update and cite. Many production systems use both.
What is a vector database?
A vector database stores embedding vectors and finds the nearest ones to a query vector quickly, usually with an approximate index. Examples include FAISS (a library), pgvector for PostgreSQL, Pinecone, Weaviate, and Chroma. Most also store each chunk's text and metadata so results can be filtered by source or date.
Does RAG eliminate hallucinations?
No. Retrieval can return the wrong chunk, the right chunk can be missing from the index, and the model can still blend retrieved text with what it memorized. RAG lowers the rate of made-up facts and makes them easier to check, but it needs evaluation like any other system.
Do long-context models make RAG unnecessary?
Not for large collections. Even a context window of a million tokens holds a small fraction of a company's documents, and every token in the prompt adds cost and latency on every request. Long contexts and RAG work well together: retrieval narrows the set, and the long window holds more of it.
Related explainers
Embeddings— Turning words and categories into vectors