The real cost of prompt caching (and when it actually pays off)

Prompt caching looks like free money on the pricing page: write tokens cost 1.25× normal, but cached reads are 0.1×. The catch is in the access pattern.

We instrumented 14 production workloads across our customer base

For workloads with >3 reads per cached prefix per 5-minute TTL, caching delivered 62–78% savings on input tokens. For workloads under that threshold — and there were more of them than we expected — caching was a net loss of 15–25% because the write premium never paid back.

The single best predictor of cache ROI was prompt-prefix entropy

Long, stable system prompts with rotating short user turns are the textbook win. Few-shot examples that change per request are the textbook trap.