AI & MCP
Managing an agent context window: budget, cuts and cache
A large window is not a strategy: you need a budget per item, a cutting order and a stable prefix for cache. Here is the split we use.
Key points
- Write the budget first: system, schemas, documents, history and answer get separate ceilings.
- Cut old tool outputs and re-read files first, then compress history.
- A stable prefix makes cache useful; a date at the top of the system prompt cancels it.
A context window is managed with a budget written per item, a cutting order decided in advance and a stable prefix the cache can reuse. Buying a bigger window is not a strategy.
Write the budget first
| Item | Tokens | Share of budget | What it holds |
|---|---|---|---|
| System prompt | 1,500 | 0.75% | role, rules, response format |
| Tool schemas | 900 | 0.45% | names, descriptions, input |
| Files and documents | 80,000 | 40% | only the files the task touches |
| Step history | 30,000 | 15% | latest results and a summary |
| Reserved answer | 8,000 | 4% | room for the model output |
| Headroom | 79,600 | 39.8% | bursts and extra calls |
The table is a design split over 200,000 tokens, not a measurement: it decides in advance what gives way when the task grows. Without separate ceilings, the first large file eats the answer space and the model truncates its output halfway.
Cut before you compress
- Old tool outputs: keep the latest result and a summary of earlier ones, not every intermediate JSON.
- Re-read files: a diff is cheaper than the whole file once the change is known.
- Logs and tests: the first lines of an error are enough; ten thousand lines rarely change the next action.
- Duplicates: the same file quoted in two steps counts once.
Order matters because each item costs a different amount to rebuild. An old output can go without losing the ability to act; an identifier cannot. Cut the data the model will not use again, never the data that explains how it got here.
Compress but keep the identifiers
When cutting is not enough, history becomes a state block: what was done, which files changed, which commands passed, what is still open. Exact identifiers stay, long descriptions do not. A summary that loses a path or an error code forces the model to redo work, and the token saving is paid back in steps.
Cache rewards a stable prefix
Cache only hits an identical prefix: system prompt, tool schemas, instructions. If the system prompt carries today's date, a randomly ordered file list or the task id, the prefix changes on every call and the discount never arrives. On deepseek-flash the gap is fifty times on input, so the order of lines is an economic decision.
What the schemas cost
node -e "const bytes=3655,budget=200000,t=Math.round(bytes/4);console.log('tool schemas',t,'tokens',((100*t)/budget).toFixed(1)+'% of budget')"The command prints tool schemas 914 tokens 0.5% of budget. Nine tools are irrelevant against 200,000 tokens; fifty tools with long descriptions change the ratio and deserve the same sum. The four-characters-per-token estimate is for decisions, not invoices.
The limitation
No split survives a task that changes shape: an agent reading a whole repository shifts everything to documents, one fixing a test spends almost all of it on tool output. Compression loses details that looked useless, and cache breaks the moment one line at the top changes. Measure the budget from the usage field, not by eye.
Where to continue
The price arithmetic per tier is in long context: real costs, the order of steps in orchestrating multi-step agents, the system prompt in prompts for agents. The catalogue behind the examples is 111,985 components and 1,556 blocks; a citable record is dqnamo agent dock.
Frequently asked questions
Is a 1M-token window better than a short context?
A short context almost always wins on cost: every step pays for the whole prompt again. A 1M-token window earns its place when the task truly needs a whole document, not as a substitute for selecting files.
When should history be compacted?
When the next step risks exceeding the budget, not when the window is full. Compact at 70-80% of the declared budget, so there is room for one more tool call and the answer.
What do I cut first?
Old tool outputs, logs and whole files read twice. The model has already used them and can rebuild them with a command; identifiers and error codes are the last thing to remove.
Does compression hurt quality?
It can, if it loses paths, function names or error messages. A good summary keeps exact identifiers and compresses only descriptions: the model must be able to continue without re-reading everything.
Why does cache never kick in?
Because the prefix changes. A date, a randomly ordered list or an id on the first line invalidates cache on every call. Keep the head of the prompt stable and move variable data to the end.