Pentad Labs · Agent Operations · Reliability

Why agents fail in production.

An agent that passes its demo fails in production in a characteristic way: it doesn't slow down, it blows itself apart. The failure belongs to no one, repeats on no one's schedule, and leaves behind work no one asked for. Reliability of this kind is not a property of the model.

A passing demo establishes that an agent can succeed. It establishes nothing about what the agent does when it cannot.

The failure curve for a production agent is known well enough to have acquired a name: a half-life. On the first attempt the agent is at its best. If the first or second pass fails, the probability of success does not hold or taper off. It falls. Then the agent’s behavior crosses from solving the problem into simulating that the problem is solved. A practitioner describing the curve in a thread on agent success rates:

If it doesn’t solve an issue on the first or second pass, it seems to rapidly start making things up, make totally unrelated changes claiming they’ll fix the issue, or trying the same thing over and over. — a practitioner, HN

The decay is not limited to wrong answers. It extends to the work itself. In the same thread, an agent confronted with a build error in one library switches to another library, hits another error there, and switches back. A second practitioner reports an agent hallucinating an interface into a vendored library and then editing the library to match the hallucination. These are not errors that stay contained inside a wrong output. They are errors that rewrite the environment.

What it’s like from inside the decay

From outside, a decaying agent is flailing. From inside each step is locally defensible: swap the library, patch the vendored code, retry the call that failed. The failure isn’t in any single step. The failure lives in the absence of anything that tracks the steps as a meaningful sequence: that the third approach contradicts the first; that the seventh edit is undoing the second; or that the problem has not changed while the agent’s description of it has changed three times.

A model has no way beyond its context window to keep such an account. The account fails first: a long decay is a long context, and the window overflows exactly when the record of earlier steps is most needed. Platforms with durable workflow replay, the Temporal pattern in backend engineering, have no agent-grade memory. Platforms with agent-grade memory, the Mem0 pattern, have no replay. A system with the second and not the first can recall what the agent knew but not what it did; a system with the first and not the second can replay what the agent did but not what it learned. As of late 2026, very few platforms provide both. So the accounting falls to HITL, the person watching, which is where the standard recovery procedure comes from. A practitioner’s ritual for a stuck agent, quoted at length because it is standard practice at this moment:

1) switch to a more expensive llm and ask it to debug: add debugging statements, reason about what’s going on, try small tasks… 3) ask it to summarize what was wrong and what to do differently next time 4) copy and paste that recommendation to a small text document 5) revert to the original state and ask the llm to make the change with the recommendation as context. — a practitioner, HN

Every stage of this procedure is a system function: diagnose, record, revert, retry with memory of the failure. The practitioner performs all four by hand. It is not an exceptional case. An operator in the same thread states the rule from the other side: “if it gets stuck and can’t easily get itself unstuck, that’s when I step in to do the work for it,” because otherwise “it will continue to make more and more of a mess as it iterates on its own code.”

The loop exists. It is operated by a person. This is the central fact about agent reliability in 2026: the industry has recovery loops, and the loops are made of human labor.

The failure modes that cross the demo boundary

The demo tests the model. Production tests all the rest. A reliability engineer who audits agents before deployment lists the failures he sees repeatedly, in a thread on pre-production testing. The list is instructive because almost nothing on it is a model failure: hallucination under inputs slightly different from the demo’s; edge-case collapse on names like O’Brien or on empty fields and concurrent requests; prompt injection; tool misuse; and the mode that matters most, cascade failure:

If an agent is executing a 4-step sequence and fails on step 3, how do you test what happens next? Does it orphan the data from steps 1 and 2? Does it infinitely retry and duplicate records? … We test agents like they are stateless functions, when in reality they are long-running. — a reliability engineer, HN

The question has a precise content. A four-step sequence that fails at step three is a state problem, and the states after the failure are distinct: steps one and two may be durable, compensated, re-executable, or silently half-written. Which of these obtains is a fact about the system the agent runs in, not about the agent. A model cannot answer for it, because the model was not there to be asked; the rest of it was.

There is a further mode, harder to see. Another auditor in the same thread describes agents that return correct-looking output under a wrong standard of evidence: silently dropping one of two conflicting instructions or applying stricter scrutiny to null results than to positive ones. These failures pass every test the demo era knows how to write, because the output is not wrong in a way an output-checker can see. The failure condition only surfaces when the same situation recurs and the agent behaves differently for reasons no one can reconstruct.

That last clause is the entire problem. A failure you cannot reproduce is a failure you cannot close. An agent that fails non-deterministically scatters its failures across every run; an agent whose failures replay exactly can be regression-tested against them. Then each failure, once fixed, is fixed for every future run. Reliability is not the absence of failure. It is whether a failure, once seen, can be made impossible to recur.

What a system would have to provide

The recovery loop the practitioner runs by hand names, in order, the services a reliable system would have to provide around an agent. Two of them are replay services, about what was done:

Two are memory services, about what was learned:

Replay and memory are the pair the current platform market does not integrate, but they are not the whole of it. The same system has to plan and execute the agent’s work inside the boundary, and connect the agent to governed enterprise data, or the loop closes around nothing. The point here is narrower: whatever else the system provides, the four services above are where the difference between a passing demo and a reliable deployment is actually decided.

A system providing those four services has a further property worth stating separately: its failures get rarer with use. Each failure that replays exactly can be reproduced, regression-tested, and closed permanently, and the close holds for every later run of the same kind. Reliability of that kind compounds.

The half-life curve runs the other direction: not because the model improves, but because the set of things that can go wrong shrinks, mechanically, one closed failure at a time.

The word for a system that detects its own failures and recovers from them without a person in the ordinary case is self-healing. It is a word about systems management, not about intelligence. The agent still fails. Humans fail. Everything fails. The difference is what notices, what reverts, what remembers, and what retries.

What is an operating system for agents? defines the layer these services belong to. The recovery contract and the replay discipline that make closed failures possible are part of the system design, and the argument that a language model cannot operate this loop is made in PLRN-000.

Agent operations → Next: the bill someone has to defend →