Learning path · Embeddings & Representation · 38
Chunking
Splitting documents into retrieval-sized pieces before embedding—balance context completeness against search precision.
Why it matters
- Bad chunking is the silent killer of RAG quality.
- Chunk size interacts with embedding model max length.
- Strategy depends on document structure and query type.
Key ideas
- Fixed-size splits
- Overlap windows
- Structure boundaries
Chunking decides what the retriever can return as atomic evidence. Fixed token windows are simple but split concepts mid-thought. Overlap reduces boundary cuts but increases storage. Compare semantic, late, and structure-aware strategies on your eval set—no universal best size. Always store metadata: source URL, heading path, page, and permissions for citation and filtering. Re-chunk when source formats change—PDF-to-Markdown migrations have invalidated more RAG systems than model upgrades ever did. Ship only after eval gates pass on representative production failures.
Updated 2026-08-09 · Full learning path