The Four Walls: Budgets That Stop a Runaway Agent
Ask an engineer how to stop an agent that won't stop, and you'll get the same answer every time: add a limit. It's the right instinct and the wrong plan. There is no single limit that catches every shape of runaway, because runaways don't have one shape — and each shape slips past a different guard for a different structural reason.
The failure mode people picture is a model producing one bad answer. That's not the interesting failure. The interesting failure is a loop that keeps making plausible, individually-reasonable calls forever. Every call, read on its own, is a call you would have approved. Only the hundredth one tells you anything is wrong — and by then the run has been busy a long time, doing work that looked like progress the whole way down.
That we build the harness rather than treat it as glue is settled ground here. This post is one level deeper: the specific mechanics of putting a floor under an autonomous loop, and why bounding one takes four independent guards instead of a number in a config file.
The insight the whole thing hangs on
Start with the guard everyone reaches for first: refuse a call the agent has already made. Deduplicate. If it asks the same question twice, it's stuck.
That guard never fires.
A runaway per-record loop calls the same tool with different arguments every time. Record 1, record 2, record 3. Message A, message B, message C. No call ever repeats itself, so a de-duplicator sees a stream of perfectly novel, perfectly legitimate requests and waves each one through. The loop isn't repeating. It's enumerating — and enumeration is indistinguishable from diligence, one call at a time.
Every wall below exists because some loop shape is invisible to the others. That's the design constraint — not defense in depth for its own sake, but four guards keying on four genuinely different things, because a loop that's obvious along one axis is featureless along the rest.
Wall 1 — the per-tool-name budget
The first wall caps calls at 40 per tool name, and it ignores arguments entirely.
That last part is the point, and it's a direct response to the shape above: the observed failure was the same tool called hundreds of times with different ids, so an identical-call guard would not catch it. Strip the arguments out of the identity and the loop becomes trivially visible. Forty reads is forty reads whether it read the same record forty times or forty different ones.
Crude on purpose. A tool name is the coarsest possible key, which is exactly why nothing about the arguments can hide from it.
Wall 2 — the aggregate budget, and the trick inside it
Wall 1 has an obvious hole: an agent that calls eight different tools thirty-nine times each never trips a per-name cap and has still done three hundred operations. So the second wall is tool-agnostic — a soft cap of 40 across all external tool operations. A mail read, a repository listing, and a knowledge search all count identically. There is no notion of a cheap call.
The clever part isn't the cap. It's what happens past it.
Past the cap, a call identity is refused once. An identical repeat is allowed through.
Sit with that for a second, because it's doing something subtle. A runaway per-record loop iterates different args each call, so no call ever repeats its identity and the loop stays refused — forever, every call, one after another. A genuinely-required call proceeds on its confirmed retry: the agent gets told no, decides the call really is necessary, asks again the same way, and gets it.
A loop cannot confirm itself, because confirming requires repeating — and the loop never repeats. The exact property that makes a runaway invisible to a de-duplicator is the property that keeps it behind this wall — the behavior that defeats one guard is what the other is built to catch. The agent isn't asked to prove intent. The mechanics of its own loop answer for it.
Wall 3 — the delivery reserve
Send, post, deliver: these draw on a separate reserve of 5 operations, argument-independent, exempt from the data-gathering cap.
Without it, the arithmetic goes badly. A run spends its entire budget reading and has nothing left to deliver the result. All research, no answer — the worst place to stop, because all of the expensive work happened and none of it reached anyone. A budget that can strand its own output is a budget with a bug in it.
There's an honest design note here too. The refuse-once-then-confirm cycle from Wall 2 was too fragile for sends. It required the agent to repeat the exact call — defeated the moment it reworded the message between attempts. And of course it rewords the message; that's a reasonable thing to do when you've been told no.
A read is idempotent enough to retry; a send is not. Different semantics, different budget. When a mechanism's assumptions don't hold for a class of tool, the answer is a second mechanism, not a stretched first one.
Wall 4 — the query-loop guard
The last wall caps at 10 and keys on (file_ref, normalized_sql). A
normalizer strips string and number literals down to a skeleton, so WHERE ts='A' and WHERE ts='B' collapse to the same signature.
This catches the loop that both other guards structurally miss. A per-tool-name count sees one tool used a reasonable number of times. An identical-call guard sees ten distinct queries. Neither is wrong. The loop is simply invisible along both of their axes — and visible immediately along a third: same query shape, different literal, over and over.
Normalization is what makes an enumerating loop's signature stop moving.
| Wall | Keys on | Catches | Can't see |
|---|---|---|---|
| Per-tool-name (40) | Tool name only | One tool hammered, any args | Many tools, each under cap |
| Aggregate (40, soft) | Call identity, all tools | Broad enumeration across tools | Nothing — but it only refuses, it can't end a run |
| Delivery reserve (5) | Send-class tools | Budget starvation before delivery | Data-gathering loops (by design) |
| Query loop (10) | (file_ref, normalized_sql) | Same query shape, rotating literals | Loops that vary query structure |
The hard stop above them
All four are soft. They refuse; they don't terminate. Above them sits a single hard aggregate cap on external tool operations — tunable per deployment, and setting it to 0 disables it.
The mechanism is worth showing because it's clean. At the cap, the harness
injects a terminal assistant message with no tool calls. The router reads
that marker and routes to __end__:
# At the hard cap: don't raise. Speak, then let the router do its job.
if hard_cap and external_ops >= hard_cap:
state.messages.append(
assistant_message(hard_stop_notice(external_ops, hard_cap))
)
return state # note: no tool calls attached
def route(state):
last = state.messages[-1]
if last.tool_calls:
return "tools"
return "__end__" # a terminal message is already the end shape
No exception. No special termination path. The run ends at the graph's normal completion shape — the same one a successful run uses — because "an assistant message with nothing left to call" already means finished. This mirrors a precedent in the same router, where a synthesis-complete message ends the run identically. The hard stop didn't need new machinery. It needed to say the thing the router already understood.
And the terminal message is written for the user, not the log:
I stopped after {n} tool calls — this run hit its hard limit of {cap} tool
calls, which guards against a runaway loop. I've summarised what I gathered so
far; if the task genuinely needs more tool calls, break it into smaller runs
or raise the agent's tool-call budget.
Error-as-data: the principle underneath all of it
None of the four walls work as exceptions. Refusals are returned as tool errors the model reads and re-plans against.
The governing rule: a refusal must never read like a success. Every refusal message states explicitly that the call did NOT execute.
This sounds fussy and isn't. A guard that silently returns nothing teaches the model that the tool is broken — and a model that believes its tool is broken does exactly what you'd do: retry it, try a neighbouring tool, work around the outage. You've converted a budget into a stimulus for more calls. A guard that explains teaches the model to change strategy, which is the only outcome you wanted. The refusal is an input to the next planning step, so it has to be written like one.
Metering by construction
A tool is metered if its canonical name isn't in the local-tools set. That's the whole rule — external ops get counted uniformly, and nobody maintains a per-vendor list that goes stale the day someone adds an integration.
Two details make it hold. A canonicalizer collapses the two surface spellings of the same tool — the function-call form and the code-mode form — into one identity, so an agent can't dodge a budget by switching call styles. And synthetic messages the harness injects itself never count; the meter measures the agent, not the scaffolding around it.
The wall that's soft on purpose
One budget is deliberately not a wall. The turn budget advances in steps of 50 — 50, then 100, then 150 — and injects a warning the agent is free to ignore. It gates nothing.
That's not a weaker version of a hard cap. It's a different tool. A soft signal shapes behavior; a hard wall guarantees termination. A warning lets a still-reasoning agent wrap up on its own terms, which is a better ending than any forced stop — but it promises nothing, because an agent that ignores it just keeps going. A hard cap promises termination and can only deliver it bluntly. You need both, and you need them separate: a soft signal that gates is a hard wall with worse ergonomics, and a hard wall that warns is a wall that doesn't work.
A seam worth naming
Only the turn budget reaches the box today. Token and wall-clock budgets are computed but not enforced in-box — the outer bound on wall-clock is the sandbox's own lifetime. That's a real gap, and we'd rather write it down than let the diagram imply coverage the code doesn't have.
The four walls are the shape of the argument, though, and the shape generalizes. The question is never "what's the limit?" It's "what does this loop look like, and which key makes it stop moving?" Answer that four times and you have something that holds.
Written by CatalEx Engineering. We build the AI operating layer for AI-native companies — one platform to build, deploy, and run AI agents in production. More at catalex.co.