Agents need a flight recorder, not a firewall.

The agent did something you did not expect, again. It moved money, changed a record, sent a message, deleted a row. You go to the logs to find out why, and the logs are green. Every step reports success. Nothing in them tells you what the agent saw when it decided wrongly, and nothing lets you replay the decision to reconstruct why it went the way it did. The run is over, and the self-account the agent left behind is the only account there is.

This is the problem an audit trail exists to solve: when software acts on your behalf, the record of what it did has to exist outside the software’s own account of itself. The agent’s logs are the agent’s testimony. Testimony is what a witness says happened. But evidence is different. Evidence is what you can check without taking the witness’s word for it. A green log is testimony wearing the clothes of evidence: it looks like proof and acts like a claim.

A log describes a run but doesn’t reproduce one

This is the distinction the word audit quietly assumes and most logging quietly fails. A description is a sentence about an event. A reproduction is the event again, under your hands, producing the same result it produced the first time. You can read a description and still not know whether it is true. You run a reproduction and find out for yourself, first-hand.

Consider two systems side by side. In the first, the agent ran, the logs are all green, and the production database is intact. In the second, the agent ran, the logs are all green, and the production database has been deleted. As evidence of what the agent did, these two systems are identical. The logs say the same thing in both. If the logs were evidence, they would distinguish the case where nothing broke from the case where everything did. They do not, because they were never evidence. They were the agent’s account, and the account reads the same whether it’s false or it’s true.

What an auditor actually needs

The record an auditor needs has three properties.

First, it’s reconstructible. From the record you can rebuild what the agent saw at the moment it decided, not a summary written afterward. The inputs to the decision are retained, so the decision can be examined against the same things, the exact same state-of-the-world, the agent had in front of it.

Second, it’s tamper-evident. If the record is altered after the fact, the alteration shows. A record you can quietly edit is testimony again, because it says whatever the party that controls it last wanted it to say. Evidence that can be revised without a trace is not evidence: that’s merely a draft.

Third, it’s replayable. You run the retained inputs back through the same logic and reach the same verdict, every time. Replay is what turns a description into a reproduction: the decision happens again, in front of you, and you check it rather than trust it.

These are the properties of an agent’s flight recorder, not a firewall. A firewall decides what may happen and then forgets. A flight recorder retains what did happen in a form that survives the very same crash that it exists to explain after the fact.

The reason you cannot explain the incident is that you have a firewall and you need the recorder.

The record is a property of the runtime, not a report added afterward

None of this can be bolted on after the fact.

A reporting layer added to a system that was not built to be replayed can only report what the system chose to emit; now we’re back to testimony with better formatting.

1. Reconstruction requires that the inputs to each decision were retained as the decision was made.
2. Replay requires that the logic is deterministic, so the same inputs reach the same verdict rather than a plausibly similar one.
3. Tamper-evidence requires that the record was sealed as it was written.

Each of these is a property that the agent harness or runtime has or does not have at the instant the agent acts. You cannot make a system deterministic that has already run on nondeterministic state, and you cannot seal a record that was written in pencil.

Sure, a software team building agents can engineer a harness that does all this, but do you really want all of your software teams reinventing that complex wheel over and over?

The flight recorder and its record are what a proper agent operating system provides

All this is why the record is a property of the operating system the agent runs on, not of the agent itself or of its harness alone. An agent is a thing—neural model plus harness—that acts. What it acts within decides whether its acting leaves evidence or a draft. When the runtime owns the record, three things that are usually asserted become checkable instead.

1. Every action traces to the authority that permitted it. The agent does not claim it was allowed to act. The record shows which capability authorized the action and who delegated it, authority sealed into the decision rather than logged beside it.
2. Every decision replays to the same verdict, because the runtime is deterministic and auditable by construction, and the record is a tamper-evident chain that the party who needs to trust it can independently verify offline, without asking a vendor, or anyone else, to vouch for it.
3. The whole system runs as a closed loop that keeps its own replayable record as a condition of operating, not as an optional export. That is one of the things that makes it autonomic rather than merely instrumented: the record is not a feature it offers, it’s an integral, inseparable part of how it runs at all.

A firewall is something you buy and place in front of the agent. The record is something the runtime is.

The record is what lets you say yes

The record is retrospective. It answers the question you ask after the agent has acted: what the hell did it do this time? The same property answers a question you ask beforehand: what will I let it do?

You can admit an agent to data you control exactly to the extent that you can replay and check everything it does with that data. The record is what makes saying yes possible, which is the gate it opens.

Some common questions with answers

How do I prove what an AI agent did? You prove it by reconstruction, not by citation. Retain the inputs to each decision as the decision is made, seal the record so that any later alteration shows, and keep the logic deterministic so the decision replays to the same result. A threefold discipline that touches every part of agentic engineering and operations. Proving what the agent did then means running the record and showing the verdict, rather than quoting a log and asking to be believed.

What logging does the EU AI Act require for agentic systems? Article 12 requires high-risk AI systems to record events automatically over their lifetime and to keep those logs for at least six months, with enforcement from August 2026. The requirement is a minimum obligation, not the point. Logs retained for six months prove what the system said happened. What a regulator can act on is a record they can reconstruct and replay, which is the difference between meeting the retention rule and being able to answer the questions the regulation exists to ask.

Is an audit trail different from observability? Observability tells you the system is healthy. Stuff like latency, error rates, throughput, the shape of traffic. It’s built to answer “is it working” in the present tense. An audit trail answers “what did it do, exactly, and can you show it” in the past tense, for a specific action, to someone who wasn’t there and doesn’t trust you. Observability samples and aggregates because it is watching everything at once. An audit trail retains and seals because it may have to defend a single decision years later. A system can be fully observable and totally unable to reconstruct even one decision.

Can I add an audit trail to agents I have already deployed? Only partly and the part you cannot add is the part that matters. You can emit more logs today. You cannot retroactively retain inputs that were never captured, make deterministic a path that already ran on nondeterministic state, or seal records that were written in the open. Reconstruction, replay, and tamper-evidence are properties the runtime has while the agent acts. A system not built to be replayed can be described better, but it cannot be reproduced.