The bill someone has to defend.

Practitioners already feel the shape of the problem. A Hacker News thread asking whether the costs of AI agents are rising exponentially drew well over a hundred comments, which is what anxiety about a line item looks like. The debate inside the thread is about inference providers and margins. That debate is a distraction. The cost problem of an agent in production is not what inference costs per se. It is how much inference the agent uses, which the agent decides as it goes.

The same uncertainty appears wherever agents run without adequate instrumentation. One engineer cataloguing production failure modes puts it in a single clause: “surprise LLM bills from untracked token usage,” listed beside missing audit trails, which suggests the two are equally ordinary, which they are. (How are you monitoring AI agents in production?) A surprise bill is not a billing error. It is the system reporting, at the end of the month, decisions it made continuously and which no one managed continuously or at all.

The retry loop is a cost curve

The reliability problem and the cost problem are the same problem being surfaced in two directions. An agent in decay does not merely fail to finish its task. It bills for the failure: each churn loop is a sequence of paid calls. The worse the agent’s position becomes, the more it spends trying to escape it. The half-life curve of success rates has a price axis.

The same cost compounding runs through the architecture of the average deployment. A model call triggers a second model to check the first, which triggers a third to arbitrate. Each layer of AI supervision is another token meter running. Teams adopt this structure because it lowers the error rate, and it does, at a multiplier on every action the agent takes. Reliability bought this way is bought with spend. The spend scales with exactly the growth the business is hoping for.

The market’s answer is a fence

The present industry answer to unpredictable spend is to fence the money from outside the agent. Pre-funded API keys with hard caps let an operator hand an agent a budget it physically cannot exceed. We’ve no quarrel with it. Several startups now sell exactly this, and it’s sensible engineering in the absence of anything better.

But a fence is still just a fence. It caps the damage of a bad run. It does nothing for the price of a good one. Under a capped-budget regime, the hundredth run of a workflow costs what the first run cost, the thousandth costs what the hundredth did, and the organization learns nothing from having paid for all of them. A cost structure that can only be bounded, never reduced, is not a solved problem. It is a permanent one with good manners.

The fence also fails on four deeper grounds, which show us where real budget control lives.

1. The layer

The first is the layer. The cap sits at the payment instrument, which is the last place in the stack a decision can be controlled. By the time a spend cap trips, every expensive decision has already been made:

• the frontier model was called where a smaller one would have served,
• the retried call went out without knowing the first attempt’s fate,
• the context window was refilled with text the system already knew.

A fence can stop the bleeding. It cannot improve the clotting factor.

2. The unit and the moment

The second is the unit and the moment. A credit card controls nothing per turn, and it knows nothing in advance. Real budget control names a bound before the work begins: this turn may spend so much, this workflow may cost so much, and the system says so before the first call is priced, not after the last one settles. A bound that arrives after the money is spent is reporting. A bound that arrives before is governance.

3. The mechanism

The third is the mechanism itself. A flat cap is blind to value: it stops the next piece of work, whether or not that work was the most valuable thing the budget could have bought. Denied work might have been the run that mattered. A cap rations by arrival order, and arrival order is not an economic judgment.

4. The fleet

The fourth is scope. The bound has to hold across the fleet, not agent by agent. Per-agent caps miss the point twice. First they ignore that one tenant’s many agents share one budget line and one risk envelope; second, they ignore that the cheapest serving of the fleet’s total work is found by looking at all of it. The repeated question types, the reusable retrievals, the stable workflow shapes, live across runs and across agents. No per-agent fence can see them.

Four levers, all of them below the agent

The cost of an agent at production scale is decided by four levers. Each is a property of the system below the agent rather than of the model.

1. How much is spent, and on what. Real budget control is not a bigger fence. It is a mechanism with three properties.

2. it treats the budget as a market: a finite supply of spend, cleared across a tenant’s competing demands at equilibrium prices, so the work that runs is provably the work the budget was best spent on, rather than the work that happened to arrive first.
3. it looks ahead: each decision a workload wants to make is priced against the outcomes it is predicted to produce, and admitted, deferred, or refused before the spend exists, with the decision re-opened as the situation changes.
4. it leaves a unified record: the cleared allocation, every refusal, and the work itself land in the same ledger, so the forecast, the decision, and the invoice can be reconciled line for line.

A platform at this level of control does not watch the budget. It allocates it.

1. How many model calls. The answer is fewer than the agent would make, every time, because much of what an agent asks a model to do was never model work. Signature verification, schema validation, authorization, scheduling, replay, policy enforcement: these have specified semantics, and a stochastic answer buys uncertainty without buying generality. Reuse pulls the count down further. An agent that has performed a retrieval before, under the same conditions, does not ask the model to perform it again. A planner that has seen a workflow shape does not spend tokens rediscovering it. The memory subsystem of an agent’s platform, the half that remembers what earlier runs established, is a cost instrument before it’s anything else.

2. Which model takes the calls that remain. The discipline is simple to state and rare in practice: use the cheapest mechanism that preserves the answer. Frontier judgment belongs to the calls that need it. Competence that has already been demonstrated can be distilled, compiled, recorded, or served by a smaller local model. Serving it there is not a compromise but an engineering gain: the result of an expensive discovery does not need to repay the cost of discovery on every execution. This is learned compilation, not a return to handcrafted rules. It requires the system to test the cheaper rung against the quality the workload requires and to send work back up when conditions drift.

3. What inference itself costs. A platform that treats model calls as cost of goods sold eventually owns the serving question, too. Small models hosted natively on the platform’s own substrate, sized to the task and held to bounded latency and bounded allocation, do not inherit a hyperscaler’s margin structure. The cheapest frontier call is the one never made. The next cheapest is the one the platform serves itself.

All four levers share one requirement: the system has to observe its own runs closely enough to learn from them, and auditably enough to trust what it learned. That rules out the convenient version of the loop, which would let a model watch the runs and adjust the system from inside. A management surface that spends money has to replay exactly what it did and prove the change it proposes actually lowered the bill. The observation has to be deterministic, or the optimization is just another unauditable agent on the payroll.

A bill that can fall

Put these levers under a system that learns from its own traces and the cost curve inverts. The first run of a workflow is the expensive one. Each later run reuses more, routes more cheaply, and skips what earlier experience disproved. The bill falls as a function of use rather than rising as a function of scale.

The word for a system that measures its own performance and improves it is self-optimizing. It is a word about accounting before it is a word about intelligence. The agent does not get smarter in the marketing sense. The system gets cheaper in the auditable sense. For the person defending the bill, the second sort of smarts is the one that serves.

What is an operating system for agents? defines the layer these levers belong to. The mechanisms—call interception, routing, deterministic answers to specified questions, and no language model in the hot management path—are laid out in the system design. The research notes carry the arguments: use the cheapest model that preserves the answer, and why the optimization loop itself cannot be another model.