Prompt caching
Also called cache, cached tokens
The provider keeping a recent copy of a request's opening text, so sending the same text again costs a fraction of reading it fresh.
Models keep nothing between requests, so an app or agent sends the whole conversation every time. Reading the same long start over and over would be slow and costly, so providers keep a short-lived copy of it. When the next request begins with exactly the same text, that part is read from the copy at a fraction of the price.
2 things follow. The copy only helps while it lasts, so a long pause means the next request pays in full. And it only matches an identical start, so one changed word near the top makes everything after it count as new.
For example
An agent session re-sends its long history with every step. While the copy is still kept, most of that history is read cheaply from it.
Put it to use
Where you'll meet prompt caching in ChatGPT, Gemini and Claude, how to use it there, and the mix-up most people make.
Daily Bytes Plus is in the Android app.