Brewers made sound beer for centuries without knowing what yeast was. They kept it and reused it from batch to batch, and judged a batch by its inputs, its practice and whether it spoiled; call that record the cellar book. The microscope came later and showed them the organism. Our new research note asks what happens when both ways of knowing are turned on language models at once.
Large language models are grown, not written. A team sets up the training and the data, runs it, and gets a system whose recipe it knows far better than its result. Two research traditions try to close that gap from opposite sides, and this week we published a research note on whether they can do more than divide the work.
Two ways to study a machine nobody wrote
Interpretability opens the model. Mechanistic interpretability, the branch the note concerns, works on the weights and the internal activations: it tries to find the features a model represents, trace how they combine, and test the account by intervening on them. It is the microscope. It needs the model on the bench, which means open weights or a laboratory’s own model, and it is still expensive per investigation, though automation is bringing the cost down.
Agentic stemmatics, the subject of our main paper, stays outside. It compares the outputs of many models the way philologists compare the surviving copies of a text: as witnesses, some of which share errors because they share a source. It works on outputs alone, including closed models across laboratories, and each comparison is cheap. It is the cellar book, and the epidemiology.
Each is blind somewhere. The microscope needs the model’s weights or internal activations, which only the owner can give, and it has no settled way to say what a resemblance between two models’ internals is worth as evidence that one descends from the other. The cellar book cannot see what every witness leaves out, or behaviour that never reaches an output.
Why they might check each other
The note makes two proposals and is careful to call them proposals.
The first runs from interpretability to stemmatics. Our method’s weakest link is the natural habit: a phrasing or an error pattern that nobody planted, which one model may have inherited from another. Planted marks, such as watermarks, can be tracked through training and have been. Our paper has not established, and we have not found work that establishes, whether natural habits can be weighed as evidence of descent against the other explanations of a resemblance, on a population whose family tree is known and whose shared causes are recorded. Internal features, matched across models, could give a second measurement of the same relationships, scored against the known tree. Two measurements are not independent just because they are different: models that share data, architecture or tooling can resemble each other inside and out without one descending from the other. So each measurement is judged by how well it recovers known relationships, with controls for those shared causes, and whether the two “check each other” is a further question the note defines rather than assumes.
The second runs the other way. Interpretability has long noticed that networks trained separately often learn similar features. The philologist has a rule for that situation: two copyists can make the same easy mistake without ever having met, so a shared reading counts as evidence of a common source only when it is unlikely to have arisen twice on its own. The note completes that rule with the term it leaves out. What matters is the comparison: how probable the shared feature is if the two models are related, against how probable it is if they are not. A feature that related models share often and unrelated models almost never is strong evidence. A feature both kinds share alike is no evidence at all. A feature that is rare everywhere but rarer still among related models counts against relationship, however rare it is. This is the ordinary logic of likelihood in phylogenetics. The note’s claim is narrower than novelty: we have not found prior work that scores shared internal features of models, as an instrument observes them, by that ratio, and since our earlier searches each missed something close, the note says so and holds the claim loosely.
What the note does not claim
It reports no result. We drafted an experiment to test the ideas and put it through six rounds of review by AI models from two families other than the one we drafted with. The rounds did not converge on a design. In rounds two to five, the objections both reviewers rated blocking fell on text written to answer the round before; in the sixth, none did. That is our own classification of the records, and nobody outside our drafting lineage has audited it. So the note publishes the eight conditions a valid test would have to meet, as we currently state them, from how the relationship between models is defined to how uncertainty is computed when every pair of models shares something with every other; they may change at pre-registration, and the last revision has not had another cold read from outside the drafting family. The design itself moves to a pre-registration that will be reviewed and published before any model is run. The review records and the brief and packet each reviewer received are published with the note.
It does not claim that interpretability is solved or unsolved, that stemmatics can detect deception, or that similar internals prove descent.
Why this matters for a review panel
Our review practice asks several AI models to read the same work and treats their agreement as worth more when the models are independent. Related models share mistakes, so knowing which models are related is not an academic question for us; it decides how much a second opinion is worth. Today that comes from vendor labels and from output habits, and both are proxies; we have not measured the independence of our own reviewers. If the method were validated, a second, internal measurement of relatedness, where weights are open, could give a panel a way to notice relations the labels hide. That is the use we have in mind, and it is also why the note’s Competing interest section says plainly that we benefit if the method works.
What happens next
The pre-registration comes next, with its own review, and then, if the budget for training models with a known family tree can be found, the test. If the internal measurement turns out to carry no descent signal, because most features are convergent, that would be worth reporting too.
This note was drafted with a Claude model (Anthropic) and revised by the author; a Codex seat (OpenAI) read it cold against the research note before publication and its eight findings were applied. The research note: L. J. Soons, “Collation and the microscope: how agentic stemmatics and interpretability could check each other”, CounterProof Research, 16 October 2026, version 1.0, research.counterproof.io/note/collation-and-the-microscope.html, doi:10.5281/zenodo.23424154. Companion to Agentic Stemmatics, doi:10.5281/zenodo.22030516.