Correction, 20 August 2026. This note was published earlier today in a materially stronger form and revised the same day, after an external review found several claims it could not support. What changed, and why, is recorded at the foot of this page. The argument that survives is narrower than the one first published.
A review finds what its reviewer can find.
That sounds trivial. It is not, because of what follows from it: when a model sweeps a codebase and returns a short report, you have learned something narrow — that model, in that harness, with that prompt, on that day, surfaced these. The defects it could not see are not in the report, and by construction cannot be. There is no line at the bottom reading and here is what I am unable to notice.
Absence of findings is scoped to the run that produced it
Every model has a shape to what it notices and what it passes over. That shape is not published, not stable across versions, and — for any specific pair of models — not measured by anyone. It is also not a property of the model alone: the same weights in an exploit-oriented agent loop, with a larger sample budget or a different prompt, do not see the same things as a single review pass.
So the honest reading of a clean report is:
This reviewer, in this harness, on this day, did not surface anything.
That is a real result. It is not the same statement as the code is clean, and the distance between those two sentences is where the risk lives.
Why that is a security question, not only an epistemic one
An attacker is not bound to your reviewer. They choose their own model, harness and budget, and they can iterate until something works — because they need one true positive and you need to have covered everything.
That asymmetry is old and AI did not create it. What AI changed is the cost of generating candidates, which has collapsed. Our own note of 14 August made the sharper version of this point: the scarce resource moved from detection to adjudication. Findings became cheap; deciding which ones are real did not.
Two things follow, and the tempting version of each is wrong.
Sharing a model with your attacker is not safety. If you review with the same lineage an attacker happens to use, your blind spots overlap — but only for an identical, non-adaptive setup. Change the harness, add samples, fine-tune on a vulnerability corpus, or put a human in the loop, and the two stop having the same blind spots — not because the overlap vanishes, but because you no longer know where it is. And if it becomes known which reviewer a class of defenders relies on, the sensible attacker searches where that reviewer is weak, or uses it as a filter to check whether a candidate would pass. Shared blind spots are not a safety property: the overlap fails when the setup differs, and it fails again when someone is aiming at it.
But the models you did not run are not disjoint territory either. It is tempting to say every lineage you skip is a whole defect map handed to whoever runs it. That is false. Model errors are substantially correlated across vendors: one study across more than 350 models found that when two models both get something wrong, they agree on the same wrong answer far more often than chance, and that this persists across distinct architectures and providers (Kim et al., ICML 2025, arXiv:2506.07962). That work measured multiple-choice benchmarks and a résumé-screening task — not vulnerability review — so it does not give us the overlap for code. What it does establish is narrower: different vendors are not automatically independent of each other. The conclusion we draw from that — that nobody should assume a second reviewer doubles coverage — is our inference, not the paper’s finding.
Both of those cut against tidy conclusions. What is left is the part that holds: the difference between what one reviewer covers and what another would is real, unmeasured, and not observable from inside a single run.
What to ask before accepting “we had it reviewed”
- What reviewed this, in what harness, and how many distinct ones? “We used AI review” is not an answer. One reviewer is a data point, not a coverage claim.
- What did the reviewers disagree about, and how was it resolved? Disagreement is the only direct evidence you have that your reviewers differ at all. If a majority vote discarded the minority finding, you threw that evidence away — and not because the minority is usually right; it usually is not. Separately: agreement among reviewers nobody has ever compared tells you nothing about coverage either way.
- What is the scope of the absence claim? A report saying “no issues found” should say what was searched, by what, and what the method cannot see. A report that cannot state its own limits has not examined them.
- Who signed it? A named party who answers follow-up questions and retracts in writing when wrong is a different artifact from a tool’s output pasted into a PDF.
What we are not claiming
We have not shown that multi-reviewer review catches more real defects than single-reviewer review. The controlled experiment designed to test exactly that was retired before it ran a single case, and we have said so publicly and at length.
Nor is varying the reviewer the only way to learn about coverage. Proofs, tests, fuzzing and human specialists each bound different things, and where a cheap oracle exists it beats any panel.
The claim we do stand behind is narrow: a clean pass from one reviewer is a scoped result, and reading it as a cleanliness certificate is an inference that result does not support. You may decide the unmeasured remainder is an acceptable risk. What you cannot do is conclude from the report that there isn’t one.
What changed in this note, and why
Published this morning; reviewed the same day by an external model seat of a different lineage from the one that drafted it; revised the same afternoon. The review returned more than a dozen defects. The ones that mattered:
- The opening used the August 2026 Bitcoin Red Team campaign as a single-model example. Our own 14 August note does not establish that it was single-model — it names both Kimi K3 and GLM 5.2, and describes human verification alongside the machine sweep. So this was not a gap in someone else’s sourcing; it was this practice contradicting its own published reporting, six days later, to make a rhetorical point about somebody else’s volunteer work. The exhibit is withdrawn rather than repaired.
- It called sharing a lineage with your attacker “tolerable.” It is not, once the attacker adapts. Rewritten, with the adaptation cases stated.
- It claimed each unused model is “an entire defect map” conceded to an attacker. Correlated-error research says the maps overlap substantially. Rewritten as a set-difference, and stated as unmeasured.
- It priced multi-model review at “a weekend and a few hundred dollars” and called it “parity.” Both unsourced; the second is an empirical claim our own disclaimer says was never measured. Both removed.
- It closed by calling a single-lineage report an “unbounded claim,” which contradicts this article’s own thesis that such a report is narrowly bounded. The unbounded thing is the inference a reader draws from it. Corrected.
- It cited Kim et al. without stating what that study measured. Now stated, along with why the transfer to code review is not established.
The same reviewer was then sent the revised version and declined to sign it off — twelve of its fifteen findings closed, three still open, and two new defects introduced by the rewrite itself, including one that reopened the very pattern the rewrite was meant to remove. Those were fixed in a third pass, and the homepage summary of this argument was rewritten a second time. That sequence is recorded here because it is the point: the first draft read best, the second read well, and neither was defensible. This is the third.
The first version was more persuasive. This one is what we can defend.
The full argument, including the limits of our own method, is in Agentic Stemmatics*, doi:10.5281/zenodo.22030516 (CC BY 4.0, not peer-reviewed).*