The real cost of prompt caching (and when it actually pays off)
The break-even math on prompt caching is non-obvious. We benchmarked 14 production workloads to find when caching saves money — and when it costs more.
Prompt caching looks like free money on the pricing page: write tokens cost 1.25× normal, but cached reads are 0.1×. The catch is in the access pattern.
We instrumented 14 production workloads across our customer base. For workloads with >3 reads per cached prefix per 5-minute TTL, caching delivered 62–78% savings on input tokens. For workloads under that threshold — and there were more of them than we expected — caching was a net loss of 15–25% because the write premium never paid back.
The single best predictor of cache ROI was prompt-prefix entropy. Long, stable system prompts with rotating short user turns are the textbook win. Few-shot examples that change per request are the textbook trap.