The fleet-audit questions you can't answer

A financial-services firm ships its first production agent fleet. Two hundred agents, for underwriting. They run for a quarter without major event, though reliability is lower than hoped. Then the auditors arrive and ask one question first:

On Tuesday the 14th, agent X accessed customer record Y. Which business unit’s budget paid for that access, and under whose authority was the agent acting?

The platform team doesn’t have an answer. Not because they are sloppy, but because no component of their stack owns the two facts an answer requires. The harness that was running agent X is gone, already upgraded to a new release. Each agent’s logs are its own. The identity that was acting has been inherited from whoever pressed deploy and no one wrote that down. The budget has been spent. The roll-up invoice is the only record that it was spent.

This is what every agent-fleet deployment today is structurally set up to say. Not by negligence. By construction.

The auditor’s questions are all like that and there are lots of them.

The shape of the hole a fleet makes

An agent is a model plus a harness. The harness is the software layer between the model and the world. Its authorities are context and state, tools, execution, orchestration, verification, observability, and governance. The inventory and the definition come from Charles Holloway’s Build Your Own Harness, which remains my favorite articulation of the layer.

The defining fact about a harness, the one that makes it a harness, is that its scope is an agent run. Every authority in the inventory is stated relative to one run: what this run can see, what this run may call, where this run’s work happens, who may stop this run. When the run ends, the harness ends with it. This is not a limitation of any particular harness. It’s simply what a harness is.

And then there are the facts a regulated enterprise needs to be true of its agents:

• Identity. Which agent was acting, persistently, outliving any single session.
• Authority. On whose behalf that agent was acting, and under what permissions.
• Accounting. Whose budget paid for which run, in the enterprise’s currency, not the run’s.
• Audit continuity. What every agent saw and did with a given piece of data, across runs, across days, in one log with one clock.
• Policy stability. The rule that bounds the run is the same rule on run one and on run ten thousand, and no run can widen it.
• Cross-agent memory coherence. Two agents working the same customer see the same customer.
• Contention fairness. When two runs want the same resource, something decides which goes first, and the decision is not up to either run.

None of these facts is scoped to anything a harness has access to. Each holds or fails during every run and in the intervals between them. When I say between, I do not mean before-then-after. I mean outside the run’s authority, all the time, while the run is mid-token and mid-plan and mid-execution. The scheduler that decides whether a run continues is deciding while the run continues. The audit log appends while the model is generating. The budget decrements in the enterprise’s ledger while the planner burns tokens.

None of these is a fact the run can assert about itself, because none of them is in the run. A harness cannot govern a fact outside its scope.

So the seven facts above either have some other owner or they’re ungoverned. Ungoverned has names that enterprise practitioners will recognize. Silent policy drift. Unauditable spend. Uncontrolled state. Not crashes. Findings.

The harness layer is critical but harnesses are interchangeable

In September, Pan, Yang, Arabzadeh, Chiang, Stoica, and Zaharia published HarnessTax: twenty-one model-harness pairs across seven models and three harnesses, Claude Code, Codex CLI, and a minimal open-source harness called Pi, on SWE-bench Lite and Terminal-Bench 2.0. Three findings matter here.

Harness choice moves task success by two to five percentage points, and moves token cost by up to five times. Pi, a harness with four tools, sits on the Pareto frontier on both benchmarks. The elaborate harnesses don’t have higher success rate. And in nine of twelve comparisons, the best result came from a harness that was not the model provider’s own.

Read that again, slowly. The layer enterprises are currently shipping as their agent stack is, on the best public evidence, interchangeable. Success rate doesn’t depend on it. Only cost depends on it. The cheapest harness is the simplest.

If the harness is the variable, whatever is invariant about an agent deployment is, by definition, not the harness. The enterprise-grade properties are exactly the invariants. So they are not in the harness. They are somewhere else or they are nowhere.

Three ways you may have this covered

Maybe you do or you may not.

First, “we log everything to Splunk, or Datadog, or our observability stack.”

Logging is retrospective. Governance is prospective. Splunk records that a run overspent, after it overspent. It won’t refuse to start the run that would overspend. If you already pay for the former, that doesn’t mean you have the latter. It’s the latter an auditor asks about.

Second, “Kubernetes plus OPA already does this.”

There is a layer missing between the pod restarted and agent X’s run 47 was stopped at 80% of tenant A’s monthly allocation. That layer is not Kubernetes.

Governing a system’s running process doesn’t mean you’ve governed the agent that is riding in or on the process. Why not? Because an agent is a system process plus an identity, a memory, a budget, and an audit trail that outlives any K8s pod. Kubernetes knows what the container did; it doesn’t know any of the rest. When the pod is rescheduled to a different node, the agent’s identity and its accumulated spend must travel; the container’s scheduler has no vocabulary for either.

Third, “this is just IAM, or a service mesh, or good engineering.”

Each of those owns a slice:

• IAM knows identities but not run budgets.
• A service mesh knows traffic but not run lifecycle or audit continuity.
• Good engineering distributes the work.

None of them owns the composition of seven authorities. But the composition is the fleet-scoped thing. Auditors do not ask about slices, they ask about effective control of the business, which lives, if it exists, in the composition of slices.

What owns the seven facts? What could own them?

The thing that could possibly own identity, authority, accounting, audit continuity, policy stability, memory coherence, and contention fairness for a fleet of agents is, by definition, a thing that:

• exists while any agent runs and while none is running;
• cannot be rewritten by any agent in the fleet;
• owns facts no agent can assert; and,
• is itself one coherent thing, not two hundred separate things.

We have a name for this kind of thing: an operating system. Its scheduler, memory protection, system call table, process boundary, filesystem, audit log, supervisor mode; every one of those was invented to deal with the fact that more than one program wanted the same resources. Not because the resources were big or numerous, though these are complications. Because the number of programs was greater than one.

But now there are two agents in the enterprise, then two hundred, then twenty thousand.

That is the whole argument. There is a layer. It has a fixed shape. The shape is not new. Enterprises will either build it on purpose or discover it on an audit finding. The second is the more expensive way to learn.

For the structural version of this argument, why a harness cannot grow into the thing that owns these facts no matter how many authorities are added, see An agent fleet needs a new kind of OS, not a bigger harness.