Each definition below is written to survive being quoted by itself, because that is how definitions travel. Limits are inside the sentence rather than in the paragraph after it. Where we mean something narrower than the market does, the narrower thing is what is written. Method terms link to the article that works them through.
Index
A: Adjudication · Adversarial code review · AI code assurance · AI code due diligence · AI code review
C: Candidate finding vs confirmed finding · Collation · Compile-proof · Consensus-as-oracle · Correlated errors · CPR report · CRA test report
D: Decidable question vs judgement question
F: Finding · Fix verification
I: Independence
O: Oracle
P: Pinned revision · Plausible-only · Positive and negative control
S: Scope · Secure code review · Severity vs evidence strength · Source trace
T: Technical debt from AI assistance · Test · Typed-decision model
W: Withdrawal log · Witness and testimony
Adversarial code review
Reviewing source code with an attacker’s question in mind (how could this be made to do something it should not), as distinct from reviewing for correctness or maintainability. Some tools now use the phrase for an arrangement in which one model reviews another model’s output; we use it for the older and wider sense, and what we add to it is stated in the entries for oracle, adjudication and independence below. The long form is Adversarial code review: what the term leaves out.
AI code assurance
A descriptive umbrella for practices aimed at establishing what AI-assisted code actually does, not a standard, a certification or a product name. An umbrella term, used on this page to cover the category rather than to name a standard.
AI code review
Automated review of a diff or codebase by a language model, producing candidate findings, usually with severities attached. Tools in this class are useful instruments and we run them; what they produce is input to a review rather than an adjudicated record of one. See A finding is not a verdict. Adversarial code review: what the term leaves out works through what this class of tool does and does not supply.
AI code due diligence
Assessment of an AI-assisted codebase by a party outside the team that built it, commissioned before an acquisition, an investment or a major integration. The question it adds to a conventional technical review is whether anyone outside the building team has established what the code does, at a named revision, with the evidence written down.
Secure code review
Reading source for security defects at a named revision, as distinct from testing a running system. Penetration testing exercises a deployed instance; the two answer different questions and a serious programme uses both.
Finding
One reviewer’s report that a defect exists, a claim to be established against the code, not a conclusion to be counted. It carries the reviewer’s habits and prompt alongside whatever is true of your codebase. See A finding is not a verdict.
Candidate finding vs confirmed finding
A candidate finding is what a reviewer raised; a confirmed finding is one established against your source with exact file and line, a reproduction path, and an impact classification; everything else is graded plausible-only. Every candidate reaches a disposition, confirmed, graded plausible-only, or discarded, and the disposition is recorded either way. See A finding is not a verdict.
Evidence rung
The label recording what evidence was actually obtained for a confirmed finding, never implying a higher one, and describing that finding’s claim rather than the codebase generally. Four rungs, weakest to strongest: source trace, compile-proof, test, live reproduction, each defined below; and for any probabilistic claim a fifth, a rate stated with an interval, from enough trials, with the per-trial logs kept, since a measured number with no kept artifact is a withdrawable number. A finding that reaches none of them is not given a rung; it is graded plausible-only, which is the other disposition. See What settles a finding.
Plausible-only
The grade for a finding that reached no evidence rung, not a claim that the question is unanswerable, only that no check settled it in this engagement, printed openly rather than buried. Some plausible-only findings could be settled by work not done in this engagement; others rest on judgement. The report says which. See What settles a finding.
Source trace
The rung recording that the code path the finding describes was read at the pinned revision, with file and line cited, which establishes what the source says, not that the behaviour is reachable in your build. It establishes what the source says, not that the behaviour is reachable in your build.
Compile-proof
The rung recording that the build was made to fail and then pass in the way the finding predicts, which establishes the compile-time claim it exercises, and nothing about runtime behaviour. It establishes the compile-time claim it exercises, and nothing about runtime behaviour.
Test
The rung recording that a test which can distinguish the defect from its absence was run, planted, observed to fail, removed, observed to pass. A test that has only ever passed has not been shown to discriminate, and does not reach this rung.
Live reproduction
The rung recording that the defect was reproduced against the pinned revision under stated conditions. It establishes the behaviour under those conditions; another configuration is a separate claim.
Oracle
A check that can settle a specified claim under stated conditions: a test, compile or reproduction shown capable of returning the other answer, or a read of a normative text for what that text says. Where one exists and has been run, it outranks any reviewer’s opinion on that claim, ours included. Mere execution is not enough: a test that cannot fail settles nothing. See What settles a finding.
Decidable question vs judgement question
A decidable question has a check that could settle it under stated conditions; a judgement question does not. Does this path execute given these inputs is decidable. How severe is this, would an attacker bother, is this trust assumption acceptable are judgement, and no volume of model output converts judgement into fact. See What settles a finding.
Independence
Three different properties, of which we claim one: independence from the code’s author. We do not claim measured statistical independence between reviewers’ errors, and we do not claim commercial independence from your market, our group builds in payment and digital-asset custody, disclosed in writing before any engagement. We claim the first and disclose the limits of the other two; we do not assess code written by us or by any company in our group; reviewer error-correlation is bounded procedurally and has not been measured for code review; and our group builds in payment and digital-asset custody, which is disclosed in writing before any engagement. What a second model does and does not supply here is the subject of Adversarial code review: what the term leaves out.
Scope
What the engagement agreed to examine, named in the report alongside what it did not. A finding’s absence from an assessment carries information only where the scope says the area was examined.
Positive and negative control
A positive control checks that an instrument can detect the thing it is looking for; a negative control checks it does not fire when the thing is absent. An instrument that has never been shown to report a hit cannot be relied on to report a miss. See What settles a finding, where an instrument of ours failed this test.
Recorded negative
A check that was run within the stated scope and did not detect the defect it was looking for, written into the report with its conditions, which is weaker than a statement that the code is sound. It tells a reader what was looked for and how, so someone who was not in the room can judge the coverage. See The attacker can run any model.
Severity vs evidence strength
Severity is a judgement about impact; evidence strength is a record of what was checked, and a finding never carries a severity its evidence does not support. Reports here state them separately, because a single blended rating hides which half is doing the work. Severity is the field with no source anchor, and the one that inflates: A finding is not a verdict.
Fix verification
Re-checking a patch against the finding it targets, at the revision that closed it, recorded as its own result, by the same bench that raised it, unless the engagement says otherwise. A closed finding names the revision that closed it, so a later reader can tell what was verified from what was merely asserted.
Adjudication
Deciding the disposition of every candidate finding (confirmed, plausible-only, or discarded) routing it to a check where one exists and to a named person where none does, rather than to a vote count. It applies to unanimous and single-reviewer findings alike, because agreement between correlated reviewers is not confirmation.
Withdrawal log
The written record of findings we retracted under the same engagement identifier, with the reason, issued unprompted to everyone who received the original. It is how a correction reaches everyone who received the original. See We withdrew our own number.
Pinned revision
The exact commit a review was performed against, named in the report. A finding that does not name its revision cannot be checked against the code it describes, whatever its other merits.
CPR report
A CounterProof assessment issued under a persistent identifier, CPR-YYYY-NNN, carrying findings,
evidence rungs, recorded negatives and any retractions, on the retention terms the engagement sets. The identifier is what your maintainers, your acquirer or your fix commits cite.
Correlated errors
The tendency of different models to make the same mistake rather than independent ones, measured in some domains and not established for vulnerability review. One study across more than 350 models found that when two models both get something wrong they agree on the same wrong answer far more often than chance (Kim et al., ICML 2025, arXiv:2506.07962), on multiple-choice benchmarks and a résumé-screening task. That this should change how you read two reviewers agreeing on your code is our inference, not the study’s finding. See The attacker can run any model.
Witness and testimony
A model’s output treated as testimony to be weighed and attributed rather than trusted for its manner, because fluency does not establish truth, wrong answers arriving in the same voice as right ones. See The illusion of the score.
Collation
Comparing multiple witnesses against each other to work out what is reliable, a process that samples and cross-checks, and that cannot recover a defect no reviewer raised. See A finding is not a verdict.
Benchmark score
A measurement of performance on that benchmark, under that benchmark’s conditions, where transfer to your codebase is a separate question the score does not answer. Contamination, correlated judges and best-of-N submission can each produce correct arithmetic about a question you did not ask. See The illusion of the score and A clean benchmark is not a clean codebase.
Consensus-as-oracle
A benchmark design in which the reference label is the average of other models’ answers rather than an independent outcome, so that a mistake the reference models share is scored as truth. Averaging two judges dampens the errors they make separately and does nothing about the ones they make together, which the correlated-errors literature finds are the large ones at the top of the field. The first disclosed instance we have studied is the workflow evaluation published by TypeSafe AI for its model Jev, whose reference is the mean of GPT-6 Astra and Claude Fable 5.1; in the twenty cases the vendor publishes, the two reference models disagree on the final action in 8 of 19. See TypeSafe Jev, graded by consensus and Field Note 1.
Typed-decision model
A model that returns only a choice, a score or a yes/no probability over options the asker supplies, and never text; the vendor term is “System One model”. It can signal that no offered option fits, by spreading its probability across them, but it cannot name the option the asker left out, so whatever the question set omits stays omitted for every such witness that answers it. TypeSafe AI’s Jev, released in September 2026, is the first public model of this kind by its vendor’s description. Not a criticism of the design, which suits routing and triage; a statement of what its outputs can and cannot be evidence of. See TypeSafe Jev, graded by consensus.
CRA test report
Security-testing evidence prepared to support a manufacturer’s technical documentation under the EU Cyber Resilience Act: supporting documentation, not a certification of conformity, and not a formally designated document type. The Regulation requires reports of the tests carried out (Annex VII(6)), evidencing the testing duty (Annex I Part II(3)), retained at least ten years after the product is placed on the market or the support period if longer (Article 13(13)). We are not a notified body. See The standards may slip, the reporting date does not.
Technical debt from AI assistance
Future cost created by shortcomings left unresolved in AI-assisted work, not maintenance in general, and not a claim that every AI-assisted project incurs it. Whether it appears depends on the task, the model and the workflow around them. See The speed fades, the complexity doesn’t.
What this page does not claim: that applying this method establishes a codebase is sound. We have not shown that multi-reviewer review catches more real defects than single-reviewer review, the experiment designed to test it was retired before it ran, and no review establishes the absence of defects nobody surfaced. The longer argument with its limits stated is in our working paper, a draft preprint that has not been peer-reviewed: research.counterproof.io (doi:10.5281/zenodo.22030516). If we have something wrong here, we will correct it in writing on this page.