Learning path · Context Engineering & Caching · 26
Cache Miss Patterns
Common production habits that break prefix reuse—mutable system prompts, non-deterministic tool JSON, and sliding-window history.
Why it matters
- Explains why caching gains disappear after innocuous refactors.
- Guides prompt layout and serialization discipline.
- Prevents false negatives in cost projections.
Key ideas
- Mutable system prompt
- Unstable tool schemas
- Sliding history window
Three patterns routinely bust caches: injecting tenant-specific lines into the system prompt on every request instead of appending dynamic facts later; serializing tool definitions or function results with random key order or whitespace; and prepending a timestamp or sliding full-history block that changes the prefix each turn. Fix by separating stable constitution from volatile user context, canonicalizing JSON, and caching summaries instead of raw ever-growing transcripts when provider rules allow. Add CI checks that fail builds when tool schema serialization order changes without explicit approval from the platform team.
Updated 2026-08-09 · Full learning path