Learning path · Context Engineering & Caching · 24
Long Context
Models and techniques supporting hundred-thousand to million-token inputs—whole codebases, corpora, or transcripts in one shot.
Why it matters
- Enables naive "dump everything" prototypes that need production refinement.
- Changes chunking tradeoffs—not every pipeline needs small chunks.
- Still requires attention to recall bias and cost.
Key ideas
- Extended windows
- Map-reduce fallback
- Cost at scale
Top resources
- 01PaperLiu et al.
Lost in the Middle: How Language Models Use Long Contexts
Why this resource. Evidence that long context still needs structure, not dumping.
Covers in this concept
- long context
- needle-in-haystack
- 02ArticleAnthropic
Contextual retrieval
Why this resource. Techniques that make long documents usable at query time.
Covers in this concept
- contextual retrieval
Long-context models tempt teams to skip retrieval entirely. That works for bounded corpora and exploratory analysis; it fails on updating knowledge bases and strict citation needs. Hybrid patterns prevail: retrieve candidates cheaply, then let a long-context model read a curated superset. Monitor spend—linear input growth hurts—and validate that answers use distant passages, not only headers. Benchmark end-to-end faithfulness on whole-corpus prompts versus retrieve-then-read; long context is not a free substitute for search.
Updated 2026-08-09 · Full learning path