The Review Is Too Late: Block the Diff Before It Exists

An agent is asked to add a CSV export to an invoicing service. It writes the change in four minutes, pulls in a small package to read and write the files, opens a pull request and picks up its next task. Eleven minutes later the dependency scanner in the pipeline flags that package: it is not on the approved list, and nobody on the team has heard of it. By then a reviewer has opened the diff, and the agent has written two more changes on top of the first. What the scanner could have prevented is now a clean-up.

Illustrative scenario: a fictional composite written to make the point concrete. It does not describe a real organisation, and its figures are not reported outcomes.

The scanner did its job. The trouble is when it did it. Every one of those checks was placed for a person who writes a diff over an afternoon, and it has not moved since the writer changed.

A finding after the diff is not a guardrail

In a conventional pipeline, continuous integration is the first automated check a person’s change meets. If a job fails, the author hears about it before anyone else spends time, and the cost of the late placement is small.

An agent changes that arithmetic. It can draft a wrong change faster than a CI job reaches the front of the queue, and it will not wait for the result. By the time a scanner flags the dependency, three things already exist: the diff, possibly a reviewer’s attention, and the agent’s belief that the task is done. A control that fires after all three is an incident report with good timing. It tells you what went wrong. It does not stop it.

Move the rules, do not write new ones

The fix is not a new rule set. Most teams already have the rules. The fix is to run them at a different point: before the change is proposed, at the one place between the agent and the repository. Call it a write gate. A change either passes it and becomes a pull request, or it does not exist.

Three checks are enough to start, and each one is sufficient on its own to fail the change.

Dependencies. Every package the change adds or bumps, including whatever a bump pulls in underneath, is compared with an explicit allow-list. Looking familiar does not count, and neither does having been fine last year.

Secret-shaped strings. The change is scanned for the shape of a credential, whether or not it is real, and for terms the task’s grant says have no place in code this agent is writing. A false positive costs a few seconds of a person’s time. A real secret in the history costs a rotation and a search of everything that cloned it.

A short list of banned patterns. Only those that are quick to check and costly to ship: a shell command assembled by joining strings that came from outside, for example. This is a filter, not a static analyser, and it should stay small enough that someone can explain every entry.

A fourth question is not about the code at all: is the grant the agent is working under still valid? An expired or out-of-scope grant should stop the change before anything is read.

And the gate fails closed. A change with no result is not a change that passed, so an empty or half-checked diff is blocked by omission, the same discipline the Deterministic Shell applies everywhere else.

Why an allow-list does not need to know what is malicious

Dependencies are the check worth building first, because models invent them. A USENIX Security 2025 paper tested 16 code-generating models on 576,000 code samples and found that, on average, at least 5.2 per cent of the packages recommended by commercial models and at least 21.7 per cent of those recommended by open-source models did not exist. Across the study the authors counted 205,474 unique invented package names.

A name that does not exist today can be registered by someone else tomorrow. An allow-list handles that without needing to be clever, because it treats three cases identically: the invented package, the real but unapproved package and the hostile one. All three are outside what was granted. The gate does not have to be right about intent. It only has to be right about membership.

One caution. That study measured code samples under its own conditions. A coding agent with tool access and a registry lookup may behave differently, and I have not measured your estate or mine. Treat the figures as a reason to check, not as a forecast.

A block is not a failure

A blocked change is the guardrail doing its job, so treat it as one. The reason goes back to the agent at once, in words it can act on: csv-streamer-pro is not on the allow-list. The agent then has two honest moves under its grant. It can revise the change so that no new dependency is needed, and the language’s standard library is often enough. Or it can ask a person whether the list should change.

Two bookkeeping rules follow. A blocked change did not count as a run of the action, so it is counted neither for nor against the track record that lets an agent work with less supervision. And every block is still logged with the task attached, because three blocks from one agent in a week is information about that agent, its instructions or the list. That is the pattern the figure ends on.

What the gate does not do

It does not replace CI, tests or review. It checks only what can be decided by code before the change exists. It will not find a logic error, a slow query or a design that is simply wrong, and an evaluation-first pipeline and a person reading the change still have to do that work.

It is also worth nothing if there is a second door. An agent that can push to a branch directly, or run an install command in a shell, walks around the gate. The gate has to be the only path from the agent to the repository, which is a job for harness permissions rather than for the gate itself. Saltzer and Schroeder called the principle complete mediation in 1975, and it applies to a pull request exactly as it applies to a file.

And an allow-list has an owner or it rots. An unowned list turns into either a rubber stamp or a blockade. I have not measured how many blocks a typical team would see in a first week, so do not plan staffing from this article.

What to do on Monday

1. Count the near misses. Take the last twenty pull requests an agent opened for you and list every dependency they added. Note how many an allow-list, had you had one, would have stopped.
2. Find the first point of visibility. Locate the moment an agent’s change first becomes visible to anyone or anything: a branch, a push, a pull request. That is where the gate belongs.
3. Move one rule. Take the check that is easiest to relocate, usually the dependency check, and run it at that point. Make a failing result return to the agent rather than to a dashboard a person reads later.

Key takeaways

• A CI-time scanner finds problems after the diff, a reviewer and the agent’s sense of completion already exist, so it works as an incident report rather than a guardrail.
• The same rules, moved to before the change is proposed, become a guardrail without becoming a new rule set.
• An allow-list is the right shape for dependencies because it checks membership, not intent, and so catches invented, unapproved and hostile packages alike.
• A blocked change is not a failed run: return the reason to the agent immediately, keep it out of the track record, and log it anyway.
• The gate only works if it is the sole path from the agent to the repository, and if someone owns the list.

Sources and further reading

• Spracklen, J., Wijewickrama, R., Sakib, A. H. M., Maiti, A., Viswanath, B. and Jadliwala, M., “We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs”, USENIX Security Symposium 2025. https://www.usenix.org/conference/usenixsecurity25/presentation/spracklen
• Saltzer, J. H. and Schroeder, M. D., “The Protection of Information in Computer Systems”, Proceedings of the IEEE 63(9), 1975. https://doi.org/10.1109/PROC.1975.9939

Where the book takes this further

The Contextual AI-Native SDLC, a forthcoming book in the series, builds this gate in working code, shows how it sits beside the grant that scopes each task and the record of how much an agent has earned, and works through a dependency bump that is caught before it becomes a diff. See the books page for its status.

This article is part of CAE Field Notes, a series accompanying the book, The Contextual Agentic Enterprise.