2 articles
A context window is not just the text you send. Tool schemas, images, thinking tokens and cached prefixes all sit in it — and the answer has to fit in whatever is left. What counts, why there are two limits and not one, what max_tokens really does, how a cut-off answer comes back as a normal 200, and how to plan the space so it doesn’t.
September 16, 2026Prompt caching stores the model’s computed state, not your text — so it saves money and latency but never a single token of window space. What a breakpoint really marks, why cache entries nest instead of slicing, and the byte-level changes that silently drop your hit rate to zero.