Sampling, the KV cache, quantization, MoE, and fast inference techniques
How a language model picks its next token
Why LLM generation caches keys and values, and what it costs
Shrinking model weights to 8 and 4 bits
Routing each token to a few experts in a much larger model
Drafting tokens cheaply and verifying them in one pass
Exact attention with far less GPU memory traffic
Serving many requests at once without idle slots
Forcing valid JSON with grammar-constrained decoding