Spend Credits Like They're Yours: Building Token-Efficient Agents
Most advice about token efficiency is prompt golf. Trim the system prompt. Cut the pleasantries. Replace the paragraph with a bullet. Shave two hundred tokens off the instructions and feel briefly virtuous about it.
It's the wrong game. Two hundred tokens is a rounding error next to a single tool result you didn't need to look at, dragged forward through every turn of a thirty-turn run. The wins that matter are structural: they come from deciding what never enters the context window in the first place. Not what gets said more tersely — what never gets said at all.
This is a practical guide to making that decision well. Four moves, each one a place where a real agent either touches something or doesn't, and then a section on what to do about it when you sit down to build your own.
Why prompt golf loses
A system prompt is written once and sent every turn. So is the tool list. So is every tool result you left sitting in the message history. The prompt is the only one of those three you can see in a text editor, which is why it's the only one anybody optimizes — and it's usually the smallest.
Here's the asymmetry that decides everything: the context window is not a budget you spend, it's a budget you re-spend. A message that enters the history on turn four is present on turn five, and six, and every turn after, until something removes it. Put a 60,000-character API response in context on turn four of a twenty-turn run and you haven't made that decision once. You've made it sixteen times.
Prompt golf optimizes the constant. Structure optimizes the multiplier.
The four moves
| The naive version | The structural version | |
|---|---|---|
| Large results | Tool returns 60,000 chars into the message history | Preview + a file reference; the bulk is queried, never carried |
| Repetition | Fifty tool calls across fifty turns | One loop inside one code cell, one turn |
| Tool surface | Bind forty tools, ship all forty schemas every turn | Names in the prompt; fetch the schema you're about to use |
| Context pressure | Fight compaction, or hit it at full speed | Arrive at it rarely; let it run early when you do |
Every row is the same instinct applied to a different surface. Let's take them one at a time.
Move 1: keep the bulk out of context entirely
Our tool layer enforces an offload contract. Any tool result over roughly 4,000 characters never enters the message history at all. What the model gets back is an ~800-character preview and a file reference — a handle to the full result, sitting on disk, which the agent queries with SQL when it actually needs something out of it.
The instruction the agent carries is blunt on purpose: the bulk is NEVER in context; query it.
The preview is the part people notice. The consequence is the part that matters: a large result never enters the message history, and therefore never enters the checkpoint either — so it isn't re-sent on every subsequent turn.
A big result you put in context isn't paid for once. It's paid for on every turn after. This is the single most underrated fact about agent design: the message history is not a log, it's an input, and it is re-read from scratch on every single turn until something takes it out.
These reference files are read-only by instruction, and the reason is unglamorous and useful: they can run to tens or hundreds of megabytes. Opening one whole would exhaust the sandbox's memory long before it exhausted anything else. The file is not a smaller version of the result. It's the full result, somewhere the model doesn't have to look at all of it.
Move 2: loop in code, not in turns
The agent has a real Python kernel, and it's taught a three-tier ladder for deciding how to act:
- Answer directly — no tool at all.
- Call a tool directly — for ONE discrete action. Don't wrap a one-shot call in code.
- Write and run code — for orchestration: chaining, looping, bulk fetching.
The rule that does the work sits at tier three: calling the same tool across a list is a LOOP — write it once in code.
Consider fetching fifty records. Done one per turn, that's fifty round trips through the model, and — worse — fifty results sitting in the message history, re-sent on every turn that follows. Done in a loop inside one cell, it's one turn and one offloaded reference.
# The per-turn version: 50 turns, 50 results in history, 50 re-sends per turn.
# turn 1: get_record(id="r_001") -> {...}
# turn 2: get_record(id="r_002") -> {...}
# ... 48 more turns, each one carrying every result before it ...
# The in-code version: 1 turn, 1 offloaded reference, history untouched.
records = [get_record(id=r) for r in record_ids] # 50 calls, 0 extra turns
open_ones = [r for r in records if r["status"] == "open"]
summary = {"total": len(records), "open": len(open_ones)}
print(summary) # small enough to enter context — that's the point
Only summary crosses back into the message history. The fifty records stay in
kernel memory, and because the kernel persists across cells, they stay reachable
— you don't re-fetch to re-use. State carries.
The tier-two caveat is real, though, and worth respecting. A single discrete action wrapped in a code cell is worse than just calling the tool: you've added a kernel round trip and a layer of indirection to buy nothing. The ladder has three rungs because there are three right answers.
Move 3: names in the prompt, bodies on demand
Every bound tool is a permanent line item. It's in the prompt on turn one and it's in the prompt on turn thirty, whether or not the agent ever calls it.
So we split the tool surface in two. Tool names are all listed in the system prompt. Their parameter schemas are not. The agent fetches a schema only for the tools it's about to use, and it's told to batch that fetch: pass EVERY tool you're about to use in ONE call, instead of calling this repeatedly — each call is a separate step.
The name list is itself capped, at 50 names, after which it truncates with a
(+N more). That number is a judgment call, not a law of nature: long enough to
function as a menu the model can browse, short enough not to become a tax on
every turn.
The same principle governs skills. A skill is loaded on demand, which means the agent pays the token cost only when the skill is actually relevant to the turn. A skill you might need on one turn in fifty should not be present on the other forty-nine.
And here's the transferable version for anyone building on the platform: bind fewer tools. An agent with forty tools pays for forty tools on turn one and turn thirty. Not because forty is too many in the abstract, but because most of those forty were bound speculatively — "it might need to update the record" — and a speculative binding is overhead on every turn that follows.
Move 4: let the harness compact, and don't fight it
Compaction is the safety net, and it works. It triggers at a threshold — 180,000 tokens — keeps only the most recent few tool outputs in full, and reports exactly what it freed. There's a reactive path too, forcing compaction when the model itself reports an overflow the static threshold didn't predict. And rolling memory consolidation runs at around 70% of the context budget and on a turn interval: deliberately before the pressure point rather than at it, because a consolidation with room to be thoughtful is a better one.
All of that is machinery you get for free. But compaction is not free in the sense that matters: something has to read the history to summarize it, and whatever gets summarized is no longer there in full. The goal isn't to compact efficiently. It's to arrive at compaction rarely — which loops straight back to Moves 1 and 2. An agent that offloads its bulk and loops in code approaches the threshold slowly. One that carries every result forward sprints at it.
When you build your own agent
The four moves above are platform mechanics. These are the decisions that are yours, in roughly the order they'll bite you.
Bind the tools the agent needs, not the tools it might need. This is Move 3 with your name on it, and it's the one lever where you can do more damage than the harness can undo. Every speculative binding is permanent overhead on every turn of every run.
Name the deliverable and the stopping condition. An agent that doesn't know what "done" looks like spends turns discovering it — and those exploratory turns are the most token-hungry kind, because each one adds to the history that the next one re-reads. "Produce a summary table of open items, grouped by owner; stop when every open item appears once" is a brief. "Look into the open items" is an invitation to wander.
Set the KPI measurement interval to something sane. Measurement runs on every Nth run, not every run. A KPI you measure every single run means a judged evaluation every single run — and a metric that moves slowly does not need measuring quickly. Weekly-drifting quality does not need a per-run verdict.
Keep KPI slots few. There are 8. Using all 8 because they're there means judging 8 criteria on every measured run. Pick the ones whose movement would actually change what you do.
Prefer one capable agent with a narrow brief over a broad agent that must first work out what you meant. Interpretation is a real workload, and you pay for it in turns before any of the useful work starts.
Match the schedule to the decision it feeds. A schedule that runs hourly for a report nobody reads hourly is the most common waste there is. It's also the easiest to fix — nobody has to refactor anything, someone just has to admit the report is a daily report.
The mindset
Read the four moves back to back and they collapse into one instinct. Don't carry the bulk — reference it. Don't repeat a call across turns — loop it in code. Don't ship schemas for tools you aren't using — fetch them when you are. Don't sprint at the compaction threshold — arrive slowly.
Every one of those is a version of the same question: what does the model never have to look at?
The efficient agent isn't the one with the terser prompt. It's the one that touches the least.
Written by CatalEx Engineering. We build the AI operating layer for AI-native companies — one platform to build, deploy, and run AI agents in production. More at catalex.co.