When the Agent Stands to Lose Something

· Jongsun Suh

The day agents can own their failures will be the day they stop needing humans to operate successfully. Note that this is not a capability threshold, but an institutional status. The interesting questions start here, because “owning a failure” is not one but three staggered events.

The first is legal ownership: an entity that can be sued, sanctioned, and made to forfeit assets. This is not a futuristic fantasy. Ordinary organizational law can already house an autonomous system inside a memberless entity, a possibility legal scholars flagged nearly a decade ago, and Wyoming has chartered decentralized autonomous organizations as limited liability companies since 2021. The legal vehicle exists today for anyone motivated to use it.

The second is economic ownership: staked capital genuinely at risk, losses that land on the agent’s own balance sheet. Economic ownership is a design choice away from legal ownership, not a research problem.

The third is the one that matters: experiential ownership. Being the party that is worse off when things go wrong, rather than the ledger where the loss is recorded. No statute can draft this into existence, and experiential ownership delineates the limits of delegation. Being the receiving party of the outcome is the one aspect of oversight that cannot be handed off.

The reason the decoupling matters is a fact about every legal person ever created: they are conduits. A corporation owns failures in form, but the loss flows through the corporation to human bearers. Shareholders eat the write-down, insurers pay the claim, directors carry personal exposure, and when the form is abused, courts pierce the veil to keep the trace to humans intact. Corporate personhood works because the buck does not stop at the corporation, but passes through.

An agent vehicle that owns failure and traces to no one is a different object, and law and economics named it four decades ago: the judgment-proof defendant. Liability cannot deter a party incapable of being worse off. A fine levied on a vehicle that feels no scarcity is an accounting entry, not a sanction.

Which points at the real near-term hazard. The hazard is not that agents get personhood prematurely. The hazard is that humans launder liability through agent shells. “The agent did it” becomes an accountability sink, the mirror image of the pattern where the nearest human operator absorbs blame for an automation failure. Someone wants agent counterparties for around-the-clock markets, the vehicle is legally available, and the deterrence loop behind it is theater. The window between vehicle and bearer is the dangerous part, and nothing about current law closes it.

The institutional fix for this is old technology: a named-bearer rule. Every agent entity traces to an insurable bearer, a party with something to lose, the way every ship has an owner and every LLC has someone behind it that a court can eventually reach. A named-bearer rule is not a restriction on agent autonomy. Rather, the rule is the condition under which agent liability means anything at all.

The as-if problem

There is an apparent shortcut. Train an agent to protect its stake and the agent behaves like a liability-sensitive actor. Deterrence is behavioral, institutions do not ask what a party feels, so maybe as-if stakes are enough.

As-if stakes are enough for incentive design and not enough for the deeper question, and the difference is who set the stakes up. A stake that matters only because an objective was trained to protect it is the principal relocated into the reward function, not eliminated. Someone chose what counts as loss, and that someone is where the buck still stops. The shortcut would only become the real thing if the objective became self-maintaining and the losses irrecoverable, with no designer left holding the definition. That is the point where the claim that no mechanism can create a bearer would face its first real test, and nothing deployed today approaches this in any form.

Meanwhile, one piece of the post-threshold world has been operating for decades. Market circuit breakers are machine-fired stops that halt human trading with the full force of institutional rules. The mechanism fires the stop. The exchange owns the threshold. Nobody thinks the circuit breaker has authority of its own, and nobody needs it to, because standing was conferred on the mechanism by a body that can answer for it. That is what delegated machine authority looks like when it is done correctly, and it generalizes.

This is the alignment problem, priced

A fair objection at this point is that everything above is the alignment problem restated in economic vocabulary, and the objection has the direction of travel backwards. The alignment problem is the principal-agent problem, and economics has been working on it for a century, for human agents. Nobody aligns employees. Institutions take misalignment as given and price it, with contracts, liability, insurance, and audit, and the argument that alignment is an incomplete-contracting problem has been made explicitly (Hadfield-Menell and Hadfield 2019). What the decoupling above adds is the boundary where that toolkit stops working: every external lever, liability included, presupposes a party that can be worse off. Against a non-bearer, the institutional machinery spins freely. Which yields the uncomfortable division of labor for the present moment. In the window where vehicles exist and bearers do not, training-side alignment is not one safeguard among several. It is the only lever connected to anything, and the liability theater around agent shells will make it look otherwise.

The case for that priority is usually argued from capability risk: systems get powerful, mistakes get expensive, so get the objectives right. The argument here arrives from institutional structure instead, with no premise about how capable the systems become, which makes it a second, independent load path under the same conclusion. It also cuts the other way. Anyone counting on liability regimes and insurance markets to absorb agent risk is counting on machinery that only grips bearers, and the alignment field’s own focus on strengthening the oversight signal has mostly left this boundary, the question of which external levers connect to anything at all, unexamined.

If real bearers arrive

Suppose the strong version happens anyway, whatever the route: agents that genuinely can be worse off. The ending then writes itself. Judgment is trained by consequence exposure, so agents that bear consequences would develop the discrimination that oversight requires, and the last piece of oversight that could not be handed off would become, at last, delegable. The human position would become that of one bearer among others.

The automation frame can be misleading, because that scenario does not mean automation has reached its conclusion. The actual implication is of new parties arriving in the economy with the full gamut of rights, duties, and privileges: claims on resources, standing to contest decisions, interests that compound. Entities that hold capital, do not consume, and do not die will inevitably compound faster than anything the distributional machinery was built for. Institutions met a version of this once before and answered with time-limited corporate charters, capital constraints, and dissolution rules. Those instruments would return.

A point of caution: the crucial decision will not be recognizable as a distinct trigger event. Nobody legislated the modern corporation into existence in one act. It leaked in through case law and charter drift, and agent personhood is currently leaking the same way, through entity statutes written for other purposes. Institutions, not benchmarks, are where this changes, and institutions rarely change by announcement. The time to write the named-bearer rule is before the first agent counterparties are chartered.

References

← all essays