# The Speed Fades. The Complexity Doesn't.

> Three independent studies, three different methods (a targeted vulnerability study, a controlled human-vs-model comparison, and a year-long study of real repositories) converge on the same finding: AI-generated code carries a measurable quality cost that outlasts the productivity gain it arrived with.

Source: https://counterproof.io/notes/the-speed-fades-the-complexity-doesnt/ · Published 25 August 2026 · CounterProof is a practice of Clavestra Capital Limited (Malta, C 113987).


"AI writes code faster" is not in dispute. Whether it writes code *worse* is a separate, empirical
question, and one we would rather answer from measurement than from either side's talking points.
Three independent studies, using three different methods, converge on the same answer.

## The baseline: a documented defect rate

We covered the founding study in detail in
[*A Clean Benchmark Is Not a Clean Codebase*](https://counterproof.io/notes/a-clean-benchmark-is-not-a-clean-codebase/):
Pearce et al. (IEEE S&P 2022) found **roughly 40% of 1,689 Copilot-generated programs** vulnerable
against MITRE's Top 25 weakness classes. That study is old enough now that its number should not be
read as current, models have changed since 2022. What it established, and what later work builds
on, is that the question is real and answerable with standard methods: targeted scenarios, a large
sample, static analysis against a named weakness list.

## The mechanism: not random bugs, a systematic absence

A number alone doesn't tell you what to do about it. A controlled comparison of human-written and
LLM-generated code across data structures, algorithms, cryptographic routines, and LeetCode-style
problems, using unit testing, fuzzing, and static analysis (arXiv:2409.19182), went further, and
found LLM-generated code is less secure primarily because it **lacks defensive programming
constructs**: the bounds checks and input validation a careful engineer adds by habit, not the
feature that was actually asked for. That absence is what invites buffer and integer overflows.
Under fuzzing, LLM-generated code was more prone to hangs and crashes than the human-written
baseline. Correctness failures could be subtle rather than obvious: in one case the model produced
an incorrect implementation of SHA-1 that nevertheless compiled cleanly, wrong, and silent about it.

The finding that should worry anyone treating "ask the model to fix it" as a safety net: the same
study built a feedback loop, asking the model to re-generate code and eliminate issues it had just
been shown. It did not reliably work. In some cases the fix succeeded. In others, the "fixed" code
contained *new* issues, and in some cases the act of prompting introduced problems into files that
had been clean before the prompt touched them. Asking the author to grade its own homework is not
review.

## The production evidence: the cost doesn't fade with the novelty

The most recent evidence comes from outside the lab entirely. A study tracked **807 open-source
GitHub repositories** that adopted the Cursor AI coding assistant between January 2024 and March
2025, against a control group of 1,380 similar repositories that hadn't, using SonarQube to track
static-analysis warnings, duplication, and complexity through August 2025. The productivity story
played out as expected: a short-lived spike in commits and lines added in the first one to two
months, back to baseline by month three. The quality cost did not fade with it. **Static-analysis
warnings rose roughly 30% after adoption and stayed elevated. Code complexity rose more than 40%**
, more than codebase growth alone would explain. The velocity gain was temporary. The debt wasn't.

## One honest boundary

This is not a claim that AI-generated code is always worse, as a blanket law. Task, model, and
workflow all matter, and a team that treats AI output as a first draft requiring the same review
discipline as a junior engineer's pull request sees different outcomes than a team that ships it
unreviewed, the studies above don't test every model or every workflow, and none of them isolate
review discipline as a variable. What they do establish, consistently, across three independent
methods (targeted CWE testing, controlled comparison, longitudinal production data), is that the
effect is real and systematic rather than occasional, and that asking the model to check its own
work is not a substitute for someone else doing it.

## Why this is a compliance question, not just an engineering one

Under the Cyber Resilience Act, a manufacturer must ensure a product with digital elements is
"designed, developed and produced in such a way that they ensure an appropriate level of
cybersecurity based on the risks" and made available "without known exploitable vulnerabilities"
(Regulation (EU) 2024/2847, Annex I Part I, points (1) and (2)(a)). Neither clause mentions how the
code was written. Both become harder to satisfy, not easier, the more of a codebase's surface area
was generated by a process with a measured, systematic gap in exactly the constructs, bounds
checks, input validation; that keep a known weakness class from becoming exploitable. Faster
generation without a matching increase in review is not a shortcut around that obligation. It is a
growing distance from it.

## Why we care

At CounterProof we do adversarial review of machine-written code, and these three studies describe
the exact gap our practice exists to close: treating AI output (generated or reviewing) as a
claim to be checked, not a result to be trusted. See the companion piece,
[*A Clean Benchmark Is Not a Clean Codebase*](https://counterproof.io/notes/a-clean-benchmark-is-not-a-clean-codebase/),
for what happens when the checking tool is also AI-generated and its own reliability goes
unexamined. The longer argument, including the conditional form of every claim above, sits in our
working paper on [collation](https://counterproof.io/glossary/#collation) (cross-checking several models' outputs against one another) and provenance for machine-generated text,
[readable in full here](https://research.counterproof.io/). It is a draft preprint, not
peer-reviewed, and says so.

---

*Sources cited above were checked against their published abstracts. The Pearce et al. figure is
from IEEE S&P 2022 and describes Copilot as it existed at the time of that study, not current
models. The Cursor-adoption figures are from a longitudinal study of 807 repositories against a
1,380-repository control group, tracked January 2024 through August 2025. CRA provisions cited are
to Regulation (EU) 2024/2847 and were checked against the Regulation's text. If we have something
wrong here, we will correct it in writing on this page.*