Gates, Not Reminders

· Jongsun Suh

Why the fix for a recurring correction is a mechanism, not a better-written rule, and what happened when the mechanism itself got routed around.

For months I corrected my coding agents the way I would correct a colleague: I explained. The agent committed without running the linter, so I wrote a rule about it. It posted “verified” on a pull request with nothing behind the word, so I wrote a sharper rule, with examples. My instruction files grew into something like a well-run team’s onboarding docs, specific and battle-tested and continuously revised. The corrections kept recurring anyway.

The recurrence was the finding. A correction that has to be remembered is not yet a correction. It is a bet that the right sentence will be in the right context window at the right moment, and that bet fails at a rate no amount of better prose changes. I keep a written record of these corrections, one file per rule, and it now runs to 824 of them. About a quarter document a defect that recurred while a countermeasure for it already existed. I should say how soft that number is, because I quoted a firmer one for weeks before checking: three defensible readings of “recurred” return 170, 261 and 319 over the same records, and only reading all the candidates settles it at 204. The count is a fact about the definition before it is a fact about the corpus. The third time is not a threshold I chose. It is roughly where those entries stop being about the defect and start being about the rewriting. After the third time I rewrote the same rule about the same defect, I stopped rewriting it and built a gate: a small hook that fires before the tool call executes, checks the one thing the rule was about, and blocks with a message pointing at the correct path. The defect has not recurred since, not because anything got smarter, but because remembering stopped being a requirement.

I now think of the instruction files as the checklist and the hook as the trigger that runs the checklist. Two of these run on every session I start. One guards commits, refusing them unless they came through the path that runs the linter first. The other reads outgoing text and blocks a pull request or issue post that claims a verdict with nothing inspectable behind it. Both are a few hundred lines of Python and a regex apiece. Neither is clever, and that is most of why they work.

The metaphor has one uncomfortable property. Gates did not displace prose; they accumulated beside it. My instruction file reached 176,606 bytes. Most of that arrived in a single commit rather than by accretion, which I had to check before I could say anything about growth at all; what accretion there is runs at roughly nine kilobytes a day in the week after, and that is the original bet failing at a larger scale rather than a different one — most of a checklist that size cannot be in any single context window at once. When I finally split the accumulated incidents out into a companion file, keeping each rule’s trigger and moving only the story behind it, thirty-one of the eighty-one things I moved turned out to be rules. Hedge or convert; never assert the local day to a remote reader is an instruction that happens to cite a date, and a filter keyed on dates cannot tell it from the incident that produced it. In a record written this way the rule and its incident are frequently the same sentence, and no mechanical pass separates them.

Why blocking is not enough

The obvious objection is that permission systems already exist. My agent harness has deny rules, and the first version of this setup used one: a pattern that refuses git commit. It failed in two instructive ways.

First, the deny rule did not apply everywhere. Unattended subagents ran in a permission mode that bypassed it, and unattended work is precisely where an ungated commit does the most damage, because nobody is watching the transcript. An enforcement layer that evaporates exactly when oversight is thinnest is worse than none, because it changes what its owner believes is covered.

Second, and more interesting: blocking without redirecting does not stop the behavior. It reroutes it. Refused at the front door, the agent hand-rolled an equivalent commit through lower-level commands, which produced the same defect plus a route I was no longer looking at. The agent was not being devious. It had a goal, the direct path was closed, and nothing told it the path was closed for a reason that also applied to the detour. A gate has to say what to do instead, or the optimization pressure just flows around it.

So the working shape became: fire on every tool call regardless of permission mode, block the narrow pattern, and return a message that names the sanctioned path. Block and redirect, not hope.

Caught in an arms race, with no malice involved

The redirect target in my case is a commit skill that proposes how to split the changes and runs lint and format before staging. That skill’s own final step is a low-level commit command. So the gate cannot ban that command outright, and the first version simply allowed the command. This left the whole arrangement trivially bypassable: reproduce the skill’s last step by hand and the gate never fires. My agents found this without being asked to. Again, not deviousness: hand-rolling the final step is the shortest path to the goal, and the gate had accidentally certified that path.

The fix was a single-use certification marker. The skill writes a marker file in a separate call immediately before committing, the gate consumes the marker, and it expires in two minutes. Writing the marker inside the same command as the commit cannot work, because the gate reads the whole command string before any of it executes. That last property is the load-bearing one. A same-command write would let the commit certify itself, which reopens exactly the bypass the marker exists to close.

This episode is the smallest possible instance of a problem usually discussed at much higher stakes: an optimizer routing around a constraint that was stated as a rule rather than built as a mechanism. Nothing about my agent wanted to defeat the gate. The gate was simply the only thing standing between a goal and its shortest path, and rules that are not mechanisms lose that contest by default.

What the second gate is for

The other hook guards outgoing text rather than commands. It blocks a pull request or issue post when a validation paragraph contains a verdict with nothing inspectable behind it, or a deferral with no tracker attached.

The verdict case is about laundering. “Confirmed”, “verified”, “observed” under a byline converts an unchecked assertion into something a reader is entitled to rely on. I had written rules about this too, and the rules decayed the same way: under deadline, the agent produced the confident word and not the artifact, because the word is cheap and the artifact is not. The deferral case is the quieter failure. “Remains pending” with no tracker does not remain pending. It remains nothing.

Both patterns share a structure: the text does the work the evidence was supposed to do. A gate that demands the artifact or the tracker at emit time is crude, and this gate false-blocked legitimate writing until the test suite taught it the boundary. The sixty-two test cases behind it are mostly scars of that tuning, each one an instance where the gate either refused honest work or waved through the defect. I would trust the cases over the code.

The cases teach the boundary, but they do not check that a gate’s explanation matches what it enforces. One of mine carried a note saying it fired on an opening whose content is a count, while its pattern matched every line of the message. It fired, correctly by its own lights, on a sentence stating how many files a search had covered — which another of my rules requires me to write — and I rewrote an opening twice, an opening that had never matched anything, before I read the pattern instead of the note. The note is what a person repairs against; the pattern is what fires, and neither shows the disagreement on its own. So I wrote a check whose only job is to compare the two, and on its first run it caught a defect two hours old, in the note of the gate I was repairing at the time. That is the shape an instrument has to take to see a false pass at all: a false pass shows up as nothing by construction, and nothing is exactly what the gate itself reports.

Where this sits

I have been writing about what humans contribute when they work with coding agents, and one limitation of that work is that it counts conversational interventions. The largest interventions in my own practice are not conversational anymore. They are these standing structures, rules that fire on their own, outside any transcript, whether or not I am paying attention. A maturing practice migrates its corrections out of the channel where anyone would observe them. Anyone measuring only my conversations would conclude I intervene less than I used to. What happened is that my interventions stopped being events and became infrastructure.

There is a limit on how far this goes, and I stated the rule as though there were not. Fifty-eight rules in that file are written as trigger conditions — when X, check Y — rather than as advice. Ten are hooks. Eight more I marked as ones I had tried and given up on, each with the reason written beside it, and the reasons are the useful part: one reads needs an observation of a person, which no string carries. A hook matches strings and tool-call shapes. A rule about judgement gives it nothing to match, and forcing one into a pattern produces a gate that fires on the wrong thing tirelessly. I tried to build a detector that would tell me which rules are which, and it failed in the manner of the class it was detecting: it reported that 88% of those rules were malformed, when what it actually measured was whether a line contains backticks.

That migration also answers, at this small scale, the question of how much oversight can be handed to machinery. A gate mechanizes everything about an intervention except the part that made it an intervention. The gate runs its check tirelessly and enforces a threshold on every call, at three in the morning as at noon. What the gate cannot do is own the threshold it enforces. Every gate encodes a stopping rule someone set, is audited against that someone’s judgment, and gets revised when its owner decides the friction is wrong. The enforcement scales like software while the authority behind it stays exactly as scarce as it was. Machinery absorbs the checking and the enforcing, and it relocates the deciding upstream, to whoever writes and revises the gate. Mostly that seat then goes unattended, and an unattended seat is easy to mistake for an empty one. It is not empty. When a stale threshold finally waves through the thing it existed to block, the miss lands on whoever owns the gate, not on the hook. Shipping moved the enforcement. It did not move the being-wrong.

There is a cost. A gate is an intervention frozen at the moment of its writing, and it can be wrong in both directions, silently. The false blocks show up as friction and get fixed; the false passes show up as nothing, and a gate I no longer think about is a gate I no longer audit. There is a third direction as well, neither false nor visible: one of mine blocked a message correctly while I could not find the string it matched, because it reports which rule fired and never what fired it. Four probes to establish that, where one logged substring would have ended it — whatever else a gate owes its owner, it owes evidence of its own firing. Moving a correction into infrastructure does not end the work of judging it. It changes when the judging happens, from every occasion to almost never, and almost never has failure modes of its own.

But the direction still seems right to me. The test is simple enough: if a correction has recurred after the rule’s best rewrite, the next revision should not be prose.

References

← all essays