- Carl Sverre argued that AI coding agents fit code tightly to the tests they are given.
- He proposed testing general properties, such as a database never corrupting, with tools that search for counterexamples. He acknowledged the trade-off when asked about run time.
Carl Sverre of the software verification company Antithesis argued that AI coding agents tend to fit their code to the exact tests they are handed. His answer is to check general properties of the software with a tool that hunts for failures. He made the case on the Software Engineering Daily episode "Software Verification in the Age of Coding Agents," published October 15, 2026.
What was said
Sverre started with test-driven development. A person writes a test that fails, then the AI keeps changing the code until it passes. He doubted this is the right path for AI. In his words, "AI tend to hyper-focus on the goal you give them." The conversation called the result overfitting: code shaped tightly around one test.
His alternative is to test broad properties, such as "the database never goes corrupt." Antithesis runs software through many different paths and looks for a case that breaks the property. Sverre said the platform "is designed to find counter examples to whatever you give it." He argued that leaves little to overfit to. The agent is pushed toward code that is generic and correct.
Why it matters
Our reading: a green test suite written for an agent shows the agent met those checks. It does not show the code is safe. As AI writes more code that no person reads closely, the quality of the checks carries more of the risk.
Sverre drew the same conclusion about effort. As coding time shrinks, he said, teams should shift effort toward verification and specification. For managers, that means asking who defines the properties and how hard they try to break the code. For buyers, it is a fair question to put to a vendor whose software was largely written by agents.
The other side
Sverre sells this kind of testing, and the excerpts offer no independent evidence that agents overfit. That claim is his view, and the host's remarks did not test it.
The host raised the cost: unit tests give an AI near-instant feedback, while simulation runs are slower. Sverre acknowledged the trade-off. He described 15-minute runs on a single pull request, parallelism of 48, and fast-forwarded time that skips sleeps in the code. More run time gives more confidence, he said, but returns probably plateau.
The approach also relies on precise specifications. Sverre did not think markdown documents necessarily cut it. He floated regenerating whole code bases from a changed specification, and said he is curious to see how that evolves.
Written by the WebPulse Newsroom with AI assistance, and checked by our editorial review: every quotation was verified against the recording's transcript. How we use AI.
The conversation this talking point comes from
- Software Engineering Daily: Software Verification in the Age of Coding Agents (2026-10-15)
