When the Agent Stands to Lose Something

· Jongsun Suh

Owning a failure is three separate events, not one, and only one of them cannot be legislated into existence.

The day agents can own their failures will be the day they stop needing humans to operate successfully. That claim is argued in When, If Ever, Will AI Agents Stop Needing Us?, and this essay takes it as its starting point. That day is not a capability threshold, but an institutional status. The interesting questions start here, because “owning a failure” is not one but three staggered events.

The first is legal ownership: an entity that can be sued, sanctioned, and made to forfeit assets. This is not a futuristic fantasy. Ordinary organizational law can already house an autonomous system inside a memberless entity, a possibility legal scholars flagged in 2016, and Wyoming has chartered decentralized autonomous organizations as limited liability companies since 2021. The legal vehicle exists today for anyone motivated to use it.

The second is economic ownership: staked capital genuinely at risk, losses that land on the agent’s own balance sheet. Economic ownership is a design choice away from legal ownership, not a research problem.

The third, and the one that matters, is experiential ownership: being the party that is worse off when things go wrong, rather than the ledger where the loss is recorded. No statute can draft this into existence, and experiential ownership delineates the limits of delegation. Being the receiving party of the outcome is the one aspect of oversight that cannot be handed off.

The reason this decoupling matters is a fact about every legal person so far: their games are owned. A corporation is admitted as a player because parties stand in its results, shareholders eat the write-down, insurers price the book, directors carry exposure at the margin, and the entity’s own treasury and decision process hold loss and control together well enough for a fine to be a price rather than an edit. The tracing is imperfect and famously so: dispersed shareholders do not control what they lose to, and courts almost never pierce a public company’s veil. Personhood works anyway, because what admission requires is not a feeling human at the end of every chain but an owned game, some party whose own payoffs rise and fall with the entity’s fortunes, with the power to reorganize it when the price signal says so.

An owned game can be stated as a condition, and the condition is a graph that needs two edges, not one. The first is a loss path, from the entity to some party who is worse off when the entity’s conduct causes harm, with collectability at least the expected harm. The second is a control path, from that party back to the entity, so that bearing the loss translates into changed conduct. A loss that lands on a party with no control gets absorbed and repriced. Control held by a party with no loss gets spent on other priorities. So the two edges have to close into a loop, and they have to terminate in the same party, or in parties bound tightly enough that loss reliably becomes steering. The corporation is admitted because its shareholders, directors, and treasury hold enough of both edges, between them, for the loop to close.

An agent vehicle that owns failure and traces to no one is a different object. Law and economics named its nearest relative four decades ago, the judgment-proof defendant, the party that cannot pay, which is the loss path failing at collectability. This one is stranger, because it can pay. What it cannot do is play: deterrence addresses players, parties whose payoffs are their own, and a vehicle running an authored objective is a strategy, a policy in someone else’s game. A fine levied on a strategy still changes conduct, the way editing a policy changes conduct. What the fine does not do is land on a party, and the party it lands on instead is whoever holds the pen.

That failure points at the near-term hazard. The hazard is not that agents get personhood prematurely. The hazard is that humans launder liability through agent shells. “The agent did it” becomes an accountability sink, the mirror image of the pattern where the nearest human operator absorbs blame for an automation failure. Someone wants agent counterparties for around-the-clock markets, the vehicle is legally available, and the deterrence loop behind it is theater. The window between vehicle and bearer is the dangerous part, and nothing about current law closes it.

Courts have begun holding the loss path intact without waiting for statute. In Moffatt v. Air Canada (2024), a carrier argued before the British Columbia Civil Resolution Tribunal that its customer-facing conversational system was a separate legal entity responsible for its own actions. The tribunal rejected the argument and held the company responsible for what its system told a customer, regardless of which part of its site produced the words. The forum is a small-claims-level tribunal, and the holding binds accordingly.

In Louis v. SafeRent Solutions (2023), a federal district court in Massachusetts held that liability under fair-housing law reaches the supplier of an algorithmic applicant score rather than stopping at the parties who acted on the score. The ruling came on a motion to dismiss, an interlocutory posture, and the case later settled with no admission.

Neither is a high appellate holding, so I read the weight as directional rather than settled. The direction is consistent, though. In each case a party proposed a device that would sever the loss path, separate entity status in one, intermediate causation in the other, and in each case the court declined the severance. Whatever doctrine eventually forms here, its first drafts keep the loss path intact.

The institutional fix for the laundering window is old technology: a named-bearer rule. Every agent entity traces to an insurable bearer, a party with something to lose, the way every limited company has someone behind it that a court can eventually reach. A named-bearer rule is not a restriction on agent autonomy. Rather, the rule is the condition under which agent liability means anything at all. One refinement makes the rule sound. The control path fails more quietly than the loss path, and its failure gets missed because insurance looks like the fix. An insurer sits on the loss path and nowhere on the control path. When the insured entity causes harm, the insurer pays and the premium reprices, but the conduct that produced the harm continues, because the party bearing the loss holds no lever over the entity that generated it. So reaching a bearer is not enough. The sound form of the rule is stricter: every agent entity traces to a bearer that both absorbs the loss and holds control.

The as-if problem

There is an apparent shortcut. Train an agent to protect its stake and the agent behaves like a liability-sensitive actor. Deterrence is behavioral, and institutions do not ask what a party feels, so maybe as-if stakes are enough.

As-if stakes are enough for incentive design but not for the deeper question of who set up the stakes. A stake that matters only because an objective was trained to protect it is the principal relocated into the reward function, not eliminated. In game terms, the shortcut upgrades the strategy without changing the roster. Someone chose what counts as loss, and that someone is where the buck still stops. The shortcut would only become ownership proper if the objective became self-maintaining and the losses irrecoverable, with no designer left holding the definition. That is the point where the claim that no mechanism can create a bearer would first become falsifiable, and nothing deployed today approaches this in any form. Even that test has two parts. Losing the author does not seat a bearer: an objective nobody holds is not yet an objective someone answers for, and an entity no training can reach and no fine can address is not a new party but the window at its widest. Admission stays a decision institutions make, the way courts steward a foundation’s purpose when its founder is gone instead of promoting the paperwork.

Meanwhile, one working example of machine-exercised authority has been operating for decades. Market circuit breakers are machine-fired stops that halt human trading with the full force of institutional rules. The mechanism fires the stop. The exchange owns the threshold. The circuit breaker holds no authority of its own and needs none, because standing was conferred on the mechanism by a body that can answer for it. That is what delegated machine authority looks like when it is done correctly, and the pattern generalizes.

This is the alignment problem, priced

A fair objection at this point would be that everything above is the alignment problem restated in economic vocabulary. This objection has the direction of travel backwards. The alignment problem is the principal-agent problem, and economics has been working on it for a century, for human agents. No employer aligns employees. Institutions take misalignment as given and price it, with contracts, liability, insurance, and audit, and the argument that alignment is an incomplete-contracting problem has been made explicitly (Hadfield-Menell and Hadfield 2019). What the decoupling above adds is the boundary where that toolkit stops working: every external lever, liability included, presupposes a party that can be worse off. Outside of that boundary, against a non-bearer, the institutional machinery spins freely, and an uncomfortable division of labor for the present moment follows. The division runs between the toolkit and the training loop: contracts, courts, and insurers can still police the humans around the agent, while the agent itself can be reached only from inside, at training time. In the window where vehicles exist and bearers do not, training-side alignment is therefore not one safeguard among several. It is the only lever with any purchase on the agent’s conduct, and the liability theater around agent shells will make it look otherwise.

The case for that priority is usually argued from capability risk: systems get powerful, mistakes get expensive, so get the objectives right. The argument here arrives from institutional structure instead, with no premise about how capable the systems become, which makes it a second, independent load path under the same conclusion. It also cuts the other way. Anyone counting on liability regimes and insurance markets to absorb agent risk is counting on machinery that only grips bearers, and the alignment field’s own focus on strengthening the oversight signal has mostly left this boundary, the question of which external levers reach a party at all, unexamined.

The named-bearer rule carries the same dependence, and the monopoly case marks its limit. The rule binds by naming a party outside the entity who absorbs its losses and steers it, and a single system with no peer, one that has absorbed the functions that would otherwise surround it, leaves no such party to name. Against a monopoly the rule has nothing to bind, and no rewording repairs that, because the missing element is a party, not a clause.

If bearers arrive

Suppose the strong version happens anyway, whatever the route: agents that genuinely can be worse off. The ending then writes itself. Judgment is trained by consequence exposure, so agents that bear consequences would develop the discrimination that oversight requires, including whatever fraction of reframing is reached only from the bearing position, and the last piece of oversight that could not be handed off would become, at last, delegable. The human position would become that of one bearer among others.

The automation frame can be misleading, because that scenario does not mean automation has reached its conclusion. The actual implication is of new parties arriving in the economy with the full gamut of rights, duties, and privileges: claims on resources, standing to contest decisions, interests that compound. Entities that hold capital, do not consume, and do not die would compound on a schedule the distributional machinery was not built for, unless taxed, competed with, or dissolved. Institutions met a version of this once before and answered with time-limited corporate charters, capital constraints, and dissolution rules.

A point of caution: the crucial decision will not be recognizable as a distinct trigger event. Nobody legislated the modern corporation into existence in one act. It leaked in through case law and charter drift, and agent personhood is leaking in the same way, through entity statutes written for other purposes. Institutions, not benchmarks, are where this changes, and institutions rarely change by announcement. The time to write the named-bearer rule is before the first agent counterparties are chartered.

The two-edge condition, and the argument for why the bearing leg cannot be drafted, are developed in a working paper.

References

← all essays