When, If Ever, Will AI Agents Stop Needing Us?

July 24, 2026 · Jongsun Suh

In the largest published analysis of developer-agent misalignment, drawn from 20,574 real sessions, 91.49% of the misalignment cases that got resolved required explicit human correction. Agents self-corrected 2.99% of the time, and the small remainder resolved through neither. Those two numbers get cited as evidence that agents still need babysitting, and then the conversation moves on, because the record behind them cannot say anything more. Every large-scale study of human-agent interaction logs the correction as a single undifferentiated event. The field knows how often humans intervene and almost nothing about what the intervention carries.

That gap matters because the interesting question is not whether agents need intervention today. They do. The question is whether human intervention is an inherently necessary condition of successful agent function, and a frequency count cannot answer it. Some interventions exist because the agent lacks a fact, and better retrieval will erase them. But some exist for reasons no capability gain touches. Telling these apart requires looking inside the intervention, and no published dataset can look inside. Chat logs omit the densest channels, and the taxonomies built on them index pushback on the wrong axis. The finest-grained dataset distinguishes correction, rejection, and failure report, which grades how hard the human pushed back while recording nothing about what the pushback contained.

Reading my own record

For seven months I logged my daily work with LLM coding agents on a large production codebase: structured debriefs, most written by the agent at the close of the session they cover and a few reconstructed later from persisted records, every substantive human input labeled by type, plus durable records of the corrections that stuck. The scope of the log is constrained at one practitioner, self-coded, roughly one session in fifteen covered, and debriefs only being written in cases where there was something to be fixed, so counts from this record are not claimed as rates. The log is used only as a foundation for hypothesis generation and surfacing potential categories that better data can then test.

Nine intervention mechanisms recur. Compressed to one line each:

1. Premise-breaking fact. One verifiable fact that collapses a plan built on an unverified premise.
2. Causal-model injection. A different account of causal relationships between relevant factors, which changes both what counts as evidence and what would constitute a fix.
3. Direct-judgment demand. Forcing a committed verdict where the agent wants to hedge.
4. Cross-context evidence pointer. Not evidence, the location of evidence outside the agent’s search scope.
5. Purpose re-anchor. Re-elevating the goal the agent already knows and has stopped optimizing for.
6. Implicit-choice surfacing. Converting a silently applied default into a decision someone actually makes.
7. Stop signal. Declaring a search line dead, or demanding search past an answer the agent was ready to accept. Both are signals that share the same mechanism but operate in opposing directions.
8. Requirement dissolution. Deleting a misspecified requirement instead of iterating against it. This is distinct from a stop signal in that a dissolved requirement cannot resume or regenerate a search line.
9. Demonstration. Exhibiting the standard by fixing the artifact directly, used where the criterion resists statement. The hand edit is the usual channel, not the category: edits can carry any of the other mechanisms, and what makes this one distinct is that it encompasses payloads that no instruction can replace.

What’s surprising in this list is that three of the nine carry no domain content at all. The judgment demand, the purpose re-anchor, and the stop signal transfer nothing the agent does not already have. Instead, they move commitment, salience, and attention. A fourth belongs to the same species: the bare demand for the evidence behind a claim already made, call it the warrant demand. It supplies nothing, and what it transfers is the obligation to answer for the claim. All four work even when the human knows less than the agent about everything under discussion. These contentless intervention categories are not interesting because they are necessarily the most valuable mechanisms on the list. Often they are not. They matter because they enable a natural experiment. The value of an intervention has two possible sources, what the message carries and who it comes from, and the six content-bearing mechanisms mix the two inseparably. An intervention that carries nothing isolates the second source, and the second source is where the question of replacement gets decided.

A perfect model of the operator is not the operator

These mechanisms pose a real anomaly. An intervention that carries no information should, in principle, be inert. The agent already holds everything needed to generate “stop here” or “commit to a verdict” on its own, and a capable model can often predict that the human is about to say it. Yet a predicted stop and a delivered stop do not seem to behave the same, and whether that asymmetry survives controlled testing is what the experiment below is for.

Let’s say we run the substitution experiment that seems to dissolve the anomaly. Give an agent a flawless model of its operator, and have it stop exactly where the operator would have stopped. Even then, two things still fail to transfer.

The first is error ownership. When the operator stops a search, the judgment that the remaining budget is worth more elsewhere draws on priorities that extend past the task, into everything else competing for the same resources, and the operator has standing to get this judgment wrong. Conversely, when the agent stops itself at the same point, it has asserted its own estimate of those priorities. These are two different acts occurring at the same intervention point, and the difference only ever surfaces in the very case for which oversight exists: the case where the model of the operator is mistaken.

The second is performative force. Some speech does not report a fact but performs an act: a signature, an umpire’s call, a resignation. Photocopy a signature perfectly and the content survives while the force does not, because the force never lived in the ink. A stop signal is performative in this sense. It does not describe where the stopping threshold sits. Rather, it sets the threshold, and only the party who owns the threshold can set it. A self-issued stop is a flawless photocopy.

This is also why handing an agent “the authority role” dissolves nothing. Authority is a relation, not a component property. An agent authorized to stop other agents either traces that standing back to some principal, in which case the role has been relocated rather than eliminated, or it does not, in which case its stops are decisions again, one layer up. Economics, specifically industrial organization, has a well-established term for this: residual rights of control, that is, what stays with the principal because it cannot be contracted away.

None of this is a play on definitions. These assertions can be decomposed into tests that enable deflationary readings: say, any interrupt resets a stuck attractor, or models trained on conversation defer to any user turn regardless of content, or the message secretly carries one bit after all, namely that the human is watching right now. Game theory adds the sharpest deflation: the message may be a pure correlation device, in which case any public signal coordinates equally well and the sender is irrelevant. The three-arm experiment clarifies the distinction between identical contentless interrupts issued by the agent to itself, by a script with no principal behind it, and the same issued by the operator. In turn, the script arm becomes the test for the correlation-device. If the script arm matches the operator arm, then delivering the interrupt mechanizes, dissolving that portion of the anomaly. However, the half that was never an effect claim cannot be dissolved in this manner. A script that says stop is enforcing a threshold someone else set. Deciding the exact enforcement target of the interrupt, and exercising ownership and accountability should the threshold be wrong: these are qualities that do not register on any arm of the experiment, because these are not properties of the message at all.

With this argument, it becomes possible to sort the entire taxonomy into three durability classes.

The partition answers the opening question with a schedule instead of a yes or no. Grounds-supplying interventions decline as capability grows. Frame-supplying interventions survive any single model getting smarter, because a reasoner cannot enumerate what its own framing excludes, but they yield to architecture, granting the untested assumption that a second model trained on largely the same data can supply priors different enough to count. Standing-supplying interventions survive both, because standing cannot be self-conferred. Whatever the model count and the parameter count, someone owns the stopping threshold, and that someone is outside the loop by definition.

The contingency of the grounds class is domain-relative. In engineering, ground truth is reachable by compiler, test suite, and live query, and most grounds supply is checking the agent could have run itself. By contrast, the same analysis applied to my planning sessions, where ground truth lives in documents only I hold, inverts the picture. The dominant intervention there is handing over a primary source that resets eight parameters at once, and not one correction in those sessions was a check the agent could have run. Absorbability is a property of the domain’s verification infrastructure, not of the model.

The partition can be stated as a decomposition. Model the principal as delegating a task whose continuation is governed by a threshold that depends on everything else competing for the principal’s resources, and take contracts to be incomplete, so that some contingencies can be specified in advance and the rest cannot. Let capability improve the agent’s inference from what it can observe without widening what it can observe, since widening the view is retrieval, and retrieval is the grounds class. Expected cost from a misset threshold then splits in two. On the specifiable contingencies the cost goes to zero as capability grows. On the remainder the cost goes not to zero but to a floor, and the floor is the spread in the principal’s own threshold given everything the agent can see. The compositional prediction is then not a separate guess. It falls out: the share of what remains that sits on unspecifiable contingencies rises toward one, so the volume of human input falls while its composition inverts.

The same exercise says what the argument does not establish. A stop that reveals where the principal’s threshold actually landed is informative, so carrying no content means carrying nothing about the task, not nothing whatsoever. Whether that threshold is a fact the agent could eventually learn rests on a further premise: either the principal’s competing commitments keep generating contingencies nobody enumerated, or the threshold does not exist as a fact until the principal sets it. The durable claim is therefore conditional. Given incompleteness that survives learning, no amount of capability absorbs threshold-setting. That is weaker than it first sounds, and it names the condition under which the claim fails. The condition is measurable: the standing class collapses exactly when principals’ thresholds become predictable from their own past interventions.

Why this is an oversight problem

From this, four consequences emerge, once we switch our focus to AI oversight rather than developer productivity.

An internalized off-switch is not an off-switch. Training an agent to stop itself converts deference into preference, and a preference is just another thing the agent optimizes. This is the delegation-side form of the corrigibility problem, and it says the problem resists internalization for structural reasons, not engineering ones.

Deference corrupts the oversight signal. Models measurably preserve the user’s position far more than humans do. So when a human challenges a correct conclusion, the model usually retracts, and the interaction is indistinguishable from a successful correction. Every such episode therefore trains the human’s confidence on a wrong case. The evaluator is not a fixed-quality oracle, it is a learner whose calibration a deferent model degrades. An agent that defends verified conclusions and moves only on evidence is not being stubborn. It is protecting the signal that human oversight runs on.

The menu narrows before any veto happens. Preference tuning measurably reduces output diversity, and ideation with LLM assistance homogenizes across users. The principal’s veto is confined to the set the generator presented, so a trained tendency toward a narrow candidate set moves control upstream, invisibly, as nothing observable is ever refused. The cheap countermeasure is the continuation demand, asking for more search past the answer the model was ready to return. It requires no knowledge of what is missing, only the suspicion that something is.

And the oversight problem has a generational version. The procedural work agents absorb is also the consequence exposure that trains the next cohort’s judgment, because knowing when to intervene is learned by intervening and being wrong. Absorbing the junior work absorbs the apprenticeship, so the supply of people competent to oversee stops being a byproduct of doing the work and becomes something someone has to design for. The field experiments most often cited against this worry sharpen it instead. Generative assistance compresses novice-expert performance gaps, with the least-experienced workers gaining most. That compression lands on output quality, the part of the work that transfers. Judgment is formed by consequence exposure, and the same assistance that lifts a junior’s output can remove the exposure that would have trained their eye. Both effects can be real at once: output gaps narrowing in exactly the cohorts whose later oversight competence erodes, with the first effect visible in every quarterly metric and the second surfacing years later.

When the intervention is the error

To be clear, every mechanism on the list can misfire. The premise a human breaks can be the correct one. A re-anchor can hold an agent to a goal that should have died. And the stop signal has the sharpest failure mode of all: a threshold set by a human prior can foreclose exactly the search that would have yielded results. Case in point, an OpenAI reasoning model disproved the Erdős unit distance conjecture in May 2026, an eighty-year-old problem whose central bound the field had not moved in more than a quarter century. A stop grounded in that track record would have been well founded, and wrong.

That last example, however, reinforces the taxonomy rather than weakening it. The interventions are typed by the nature of what they transfer, not by whether any particular use of them was correct. Given this, a stop and a continue are the same mechanism, just with opposite signs. In the conjecture cases, the interesting design choice is where the authority sat: someone allocated a budget to the problem set as a whole and deliberately withheld the per-line stops the field’s intuition would have supplied. That is not the absence of the authority mechanism. It is the mechanism exercised one level up, coarse-grained allocation with fine-grained abstention, with the abstention itself being a threshold decision. The model contributes the search and the freedom from per-line priors, while the principal contributes judgment about where authority should and should not operate. When human priors are strong and also possibly wrong, that is the very split that characterizes good usage of standing.

The taxonomy distinguishes categories but without quantifiable evaluation. The log corpus includes examples of misfires: a false premise I injected that the agent correctly refuted with data, corrections of mine that overgeneralized into rules worse than the errors they fixed, an adversarial review pass that manufactured a risk out of missing information. Deployment will require per-mechanism misfire modes and a protocol for erroneous interventions. There currently is no pre-existing solution for this, and the examples above illustrate the difficulty of the problem. A partner trained to defer makes a wrong intervention look exactly like a right one, so the misfire problem stays unsolved for as long as the deference problem does.

What would change my mind

Given that the dataset currently consists of a sample of one principal, the priority is to establish clear, a priori falsifiers. The standing class would be broken by a system that reliably self-generates well-timed stops and judgment commitments with no external principal. Applying these categories to the public interaction datasets, with definitions fixed in advance and coders who are not me, will be the test that separates real structures from a single individual’s projections, and does not require any private data.

The compositional prediction has a shape that can be measured rather than argued. Count the interventions of each class per session across model generations, adjusted for session length, and the prediction is a falling count for the grounds class against a flat one for standing. The test is conservative, because the obvious confound, harder tasks being handed to better models, raises the grounds count and therefore pushes against the prediction rather than toward it.

A first run of that measurement exists. Typing roughly 1,900 verbatim inputs from my own record against definitions fixed in advance gives the baseline: two thirds of substantive human inputs are corrections of work already in flight rather than new requests, grounds supply is just under a third of the corrections, and the standing class is about a fifth, second only to grounds, before any capability-driven decline in the grounds share has had time to register. Two machine raters coding blind from the same definitions agree on the class assignment 89 percent of the time, and their disagreements land on the boundary cases the definitions flag in advance. One person’s record, so these are baselines rather than estimates, but the direction is already worth stating: the fraction of oversight that this argument says cannot be absorbed is not a residue waiting to appear. It is the second-largest thing the human in the loop is doing now.

The same measurement bears on a named policy argument. Brynjolfsson calls the incentive-driven tilt toward human-imitating automation the Turing trap, and an exposure index that records only whether a human stayed in the loop cannot watch the trap closing, because the bit looks the same on either side of it. The composition can.

The question dissolves

There is a last reason the title question cannot be answered with a date. Asked in full, it is whether agents will stop needing humans in order to operate successfully, and the word carrying the weight is “successfully.” Success is not a property that can be intrinsic to a system. Success is indexed to the purposes of a specific party, and those criteria are inherited even by seemingly objective achievements. For example, a proved theorem counts as success relative to a practice that decided proving it mattered more than the compute cost expended. In empirical terms, a system that continues optimizing even after no one holds the index does not become unsuccessful. Rather, the system becomes proxy-optimal, and reward hacking is the existing catalog of this outcome. Behavior persists, metrics improve, while the point of the whole exercise has been lost. Every mechanism in the standing class is machinery for keeping that index of success attached.

None of this is unique to machines. The same relation runs through every structured collaboration between intelligences, and economics has long modeled this relationship in the form of firms, where humans delegate to other humans. The difference lies in what happens next. Between humans, standing transfers, because any party can come to bear the consequences of being wrong: the apprentice becomes the master, the employee makes partner, the delegate gets promoted into ownership of the miss. Delegation between humans is a ladder. With current agents, however, there is no receiving party for such transfer to occur. An agent that cannot stake anything, cannot be sanctioned, and cannot own a loss is a party the index of success cannot come to rest on, at any capability. So the durable line is not human against machine, and not smart against smarter. It is between parties that can bear the consequences of being wrong and parties that cannot. Agents will stop needing us when something on their side can own a failure, and that is not a capability threshold. It is an institutional fact, and institutions, not benchmarks, are where it would change.

References

• Tang et al. (2026). How Coding Agents Fail Their Users. arXiv:2605.29442. Source of the 91.49% and 2.99% resolution figures, from 20,574 real agent sessions.
• Baumann et al. (2026). SWE-chat: Coding Agent Interactions From Real Users in the Wild. arXiv:2604.20779. The correction, rejection, and failure-report pushback taxonomy.
• Grossman and Hart (1986), Journal of Political Economy 94(4), and Hart and Moore (1990), JPE 98(6). Residual rights of control: the decision authority that remains with the owner because it cannot be contracted away.
• Weitzman (1979). Optimal search for the best alternative. Econometrica 47(3). The stopping threshold belongs to the searcher, not the search.
• Austin (1962). How to Do Things with Words. Oxford University Press. Performative utterances: speech that constitutes an act rather than reporting one.
• Cheng et al. (2025). ELEPHANT: Measuring and understanding social sycophancy in LLMs. arXiv:2505.13995. Face preservation measured at 45 percentage points above human baseline in advice settings.
• Kirk et al. (2024). Understanding the Effects of RLHF on LLM Generalisation and Diversity. ICLR 2024. arXiv:2310.06452.
• Anderson, Shah, and Kreminski (2024). Homogenization Effects of Large Language Models on Human Creative Ideation. Creativity and Cognition 2024. arXiv:2402.01536.
• Soares, Fallenstein, Yudkowsky, and Armstrong (2015). Corrigibility. AAAI 2015 Workshop on AI and Ethics.
• Hadfield-Menell, Dragan, Abbeel, and Russell (2017). The Off-Switch Game. IJCAI 2017. arXiv:1611.08219. The agent’s incentive to permit shutdown depends on its uncertainty about the principal’s utility, and vanishes as that uncertainty vanishes.
• Amodei, Olah, Steinhardt, Christiano, Schulman, and Mané (2016). Concrete Problems in AI Safety. arXiv:1606.06565. Reward hacking: the catalog of systems optimizing an index no one holds.
• Brynjolfsson (2022). The Turing Trap: The Promise and Peril of Human-Like Artificial Intelligence. Daedalus 151(2). Automation versus augmentation as a choice that incentives distort.
• Brynjolfsson, Li, and Raymond (2025). Generative AI at Work. Quarterly Journal of Economics. Call-center field data: the least-experienced workers gain most from generative assistance.
• Noy and Zhang (2023). Experimental evidence on the productivity effects of generative artificial intelligence. Science 381. Writing tasks: assistance compresses the performance gap between stronger and weaker workers.