LLM
How large language models work in real applications, explained step by step.
4 posts
What Fills Your Context Window, and Why Does the Answer Run Out of Room?
A context window is not just the text you send. Tool schemas, images, thinking tokens and cached prefixes all sit in it — and the answer has to fit in whatever is left. What counts, why there are two limits and not one, what max_tokens really does, how a cut-off answer comes back as a normal 200, and how to plan the space so it doesn’t.
How Does Prompt Caching Actually Cut Your Bill?
Prompt caching stores a model’s computed work, not your text. It saves money and time, but it never saves space in the context window. What a breakpoint marks, why cache entries nest, how long an entry lives, and when caching pays for itself, worked through step by step on the Claude API.
How Do You Keep a Growing Conversation Cached?
In a chat, every request is longer than the one before, so the end of the prompt keeps moving. This post follows one support bot turn by turn: where the cache markers go, what each turn really costs, the 20-position lookback, and how to shrink a long history without breaking the cache, worked through on the Claude API.
Why Isn't Your Prompt Cache Hitting?
A prompt cache that stops hitting throws no error. The answer is still right; only the bill goes up. The misses that are not bugs, the ordering mistakes that are, which changes break which part of the cache, and how to catch a broken cache before it reaches your bill, worked through on the Claude API.