AI Engineering Blog
How LLMs, RAG and databases work under the hood, explained step by step in plain language.
9 posts
- LLM
Why Isn't Your Prompt Cache Hitting?
A prompt cache that stops hitting throws no error. The answer is still right; only the bill goes up. The misses that are not bugs, the ordering mistakes that are, which changes break which part of the cache, and how to catch a broken cache before it reaches your bill, worked through on the Claude API.
- LLM
How Do You Keep a Growing Conversation Cached?
In a chat, every request is longer than the one before, so the end of the prompt keeps moving. This post follows one support bot turn by turn: where the cache markers go, what each turn really costs, the 20-position lookback, and how to shrink a long history without breaking the cache, worked through on the Claude API.
- LLM
How Does Prompt Caching Actually Cut Your Bill?
Prompt caching stores a model’s computed work, not your text. It saves money and time, but it never saves space in the context window. What a breakpoint marks, why cache entries nest, how long an entry lives, and when caching pays for itself, worked through step by step on the Claude API.
- LLM
What Fills Your Context Window, and Why Does the Answer Run Out of Room?
A context window is not just the text you send. Tool schemas, images, thinking tokens and cached prefixes all sit in it — and the answer has to fit in whatever is left. What counts, why there are two limits and not one, what max_tokens really does, how a cut-off answer comes back as a normal 200, and how to plan the space so it doesn’t.
- RAG
How Does a Text Splitter Actually Chunk Your Documents?
What chunking actually is, what makes a chunk a good one, and how a text splitter decides where to cut — including why the overlap you configure is usually not the overlap you get.
- RAG
What Is RAG, and Why Does Your LLM Need It?
A beginner-friendly introduction to Retrieval-Augmented Generation: why an LLM confidently makes things up about your data, why pasting all your documents into the prompt does not fix it, and how the RAG pipeline works, one step at a time.
- Database
How Does a Database Clean Up After Itself?
MVCC leaves a dead row version behind on every update. This is the cleanup story — why bloat slows queries down instead of just filling disk, what VACUUM and VACUUM FULL really do, how autovacuum decides when to run, why one forgotten transaction can block cleanup across the whole database, and how freezing stops 32-bit transaction IDs from silently eating your data.
- Database
How Does a Database Let Everyone Read and Write at Once?
A ground-up tour of MVCC in PostgreSQL — row versions, the hidden columns and how a tuple is really laid out on disk, snapshots, and the visibility rule — and how they let thousands of transactions read and write at once without waiting on each other.
- Database
How Does a Database Actually Store Your Data?
A deep dive into pages, slotted pages, heap files, and full table scans — the physical layer underneath every SQL query we write.