Running agents is the hard part.

Building an agent—a model plus a harness—is now a solved exercise. A framework, a model, a few tool connections, and a working demo exists by the end of the week. The demo is genuine. It is also the last stage at which the agent behaves as advertised.

In production the same agent hallucinates a library interface and then edits the library to conform with the hallucination. It handles the sample data and fails on a customer named O’Brien. It runs in March at one price and in May at a price someone has to defend in a budget meeting. When it fails partway through a workflow, a person is paged to reconstruct what it was doing.

These failures have a common shape. A new kind of program, one that reads, decides, writes, waits, spends, and remembers, is running without the right thing managing it. What follows below is five notes, one for each way the missing management surfaces.

Why agents fail in production

The pattern is common enough to have acquired a name: a half-life for the success rates of agents. An agent that does not solve a problem on the first or second pass does not grind toward the answer. It decays:

If it doesn’t solve an issue on the first or second pass, it seems to rapidly start making things up, make totally unrelated changes claiming they’ll fix the issue, or trying the same thing over and over. — a practitioner, HN

Testing practice has not caught up with the failure shape. Teams test whether the model breaks. Few test what the workflow does next:

If an agent is executing a 4-step sequence and fails on step 3, how do you test what happens next? Does it orphan the data from steps 1 and 2? Does it infinitely retry and duplicate records? … We test agents like they are stateless functions, when in reality they are long-running. — a reliability engineer, HN

The difference between failing software and recovered software is currently a person, a “human in the loop”, or HITL. The ubiquity of the acronym reflects the ubiquity of the problem. One practitioner’s procedure for a stuck agent runs as follows: switch to a more expensive model, ask it to debug, ask it to summarize what went wrong, paste the summary into a text file, revert everything, and re-run with the text file as context. The procedure works. It is a repair loop operated by hand, and it exists because nothing below the agent repairs anything.

Why Agents Fail in Production →

The bill someone has to defend

Agent cost is a standing anxiety among practitioners, even more so amongst their CFOs. The Hacker News thread asking whether agent costs are rising exponentially makes the pain points concrete. The source of the anxiety is not the price of inference. It is that the agent’s own behavior writes the check, and the amount is known only afterward:

Common failure modes: no visibility into what the agent did step-by-step, surprise LLM bills from untracked token usage, risky outputs going undetected, and no audit trail for post-mortems. — Ask HN: How are you monitoring AI agents in production?

The reliability half-life is also a cost curve, since every churn loop is billable. The industry’s current answer is to fence the money from outside: pre-funded API keys with hard spend caps, so that an agent physically cannot exceed its budget. A cap bounds the damage. It does nothing to make the next run cheaper than the last, and a system whose costs can only be capped, never reduced, is a standing charge on the organization that owns it.

The Bill Someone Has to Defend →

Who babysits the agents?

Every agent in production implies a person who restarts it, reviews it, and stands between it and its mistakes. The role is rarely in the org chart. The work always is:

Letting agents talk to each other directly was a mess. Biggest takeaway: don’t let agents pick their own subtasks. Define the task graph yourself: agents only handle the leaf nodes. — a production engineer, HN

Monitoring, where it exists, records the agent’s actions and not its reasons for acting:

Most tools record what happened (tool X was called, output was Y), but not why the agent deviated from the plan… Without causal structure in the log, you’re left correlating timestamps and guessing. — a platform engineer, HN

Builders of agent tooling have reached the same observation from the other side:

The tooling layer (desktop and console) is where the leverage is right now. The models are good enough, the harness operating them (even us humans) is what’s holding things back. — a distinguished engineer, author of Statewright, HN

People scale poorly as control loops for agents.

Who Babysits the Agents? →

Can you prove what your agent did?

The audit question arrives early and it distinguishes a log from a proof in one sentence:

How do you scope and grant an agent’s access? Least-privilege for a non-human, task-scoped actor seems like it doesn’t map cleanly. After the fact, can you actually prove what an agent did if an assessor or your ISSM asks? — a governance practitioner, r/cybersecurity

Each framework keeps its own records, in its own format, for its own agents. And in that world all that the enterprise ends up with is a pile of fragments:

Team A builds a support bot in LangGraph, Team B builds a research agent in CrewAI, and Team C writes raw Python against the Anthropic API… your audit trail is completely fractured. You can’t confidently tell a compliance officer what your synthetic workforce is doing. — a platform engineer, HN

A log records that something happened. A proof establishes that X happened, under Y authority, on Z data, and that nothing has been edited since. Regulated enterprises are being asked for the second. Those asks will turn into demands in due course. Tooling built for the demo era produces the first.

Can You Prove What Your Agent Did? →

The agent doesn’t know what you know

The other pains in this series all trace to something the platform should have and doesn’t. The last one is the widest: the agent doesn’t know what the enterprise knows. The knowledge exists, scattered across databases and documents, owned by teams, updated on nobody’s schedule, and retrieved by the agent in slices it can’t relate:

Everyone knows their slice, but the full picture lives nowhere, so bit rot just keeps creeping in. — a practitioner, HN

The industry’s answer, retrieval over the raw data estate, returns one kind of thing: the row that matches the query, the passage that resembles the prompt. What the agent needs is the enterprise’s picture assembled: the customer’s records joined to the policy that governs them, kept current because maintaining it is what the platform does.

The Agent Doesn’t Know What You Know →

Five symptoms, one cause

Let’s consider the market’s answers to the five pains, one workaround at a time.

1. For failures, a person re-prompting from a text file.
2. For costs, a budget-fence key.
3. For supervision, more operators and an external monitor.
4. For audit, a scraper per framework.
5. For context, retrieval pipelines over the raw data estate.

Each workaround is reasonable in isolation. In aggregate each rebuilds, by hand and per agent, a function that a platform exists to provide. Software that needs a human to recover it is not unreliable software plus an operator. It is unmanaged software. The management is being supplied out of someone’s working hours.

Teams that have tried to close this themselves reach the same boundary in both directions.

• Build the recovery loop, the spend bound, the supervision surface, and the proof into each agent, and the agent stops being focused on its task.
• Spread them across a sprawl of agents, and nothing in the sprawl can be trusted individually.

The first wave of agent tooling answered how to make one agent do one thing. Production asks a different question: how an organization runs many agents, reliably and accountably, without rebuilding the same plumbing under each one.

That is precisely the job description of an operating system. A layer that hosts agents as an OS hosts programs: mediating every privileged action, providing memory, identity, recovery, and accounting as services rather than rebuilds. And then managing itself while it does so. It configures itself as load changes. It heals what fails rather than paging a person. It optimizes its own costs as it learns the workload. It protects by construction what policy forbids. IBM named systems of this kind twenty-five years ago: autonomic. The idea was early then, since there was little inside the systems worth the machinery. Agents changed that.

Running agents is the hard part, and it stays hard until the management of agents belongs to a platform rather than to a person or ad hoc code. That platform is what we build: an operating system for agents that manages itself.

What is an operating system for agents? defines the layer. What is an autonomic agentic OS? defines the management discipline. The first place both are proven is the Agentic Data Enclave, where the agent that needs governing is someone else’s.