- In one Sonar demo, a coding agent introduced a security flaw that its own tests missed, and the check flagged it.
- The check rejected the agent's first fix because it broke a stricter rule the team had set. WebPulse reads this as judging outranking making.
Killian Carlsen-Phelan of Sonar argued on the AI Engineer show that a check giving the same result on every run is more reliable than an AI reviewing code, or an agent reviewing itself. His demo showed a coding agent introducing a security flaw that its own tests missed. The thesis that follows is WebPulse's, not his: when agents write the code, setting and enforcing explicit rules becomes the valuable skill.
What was said
Carlsen-Phelan showed a recording of an agent writing a small feature in under two minutes. He had told it to build a test suite. It still introduced a security problem. The agent unzipped a log file into a temporary file without setting clear permissions on it. On many servers, he said, that folder is shared.
He called this kind of flaw difficult to cover with tests and difficult to catch. A senior reviewer would usually spot it only when looking for it, he added, and sometimes not at all.
Sonar's tool, Vortex, flagged the problem, and the agent tried a fix. The tool then rejected that fix. Carlsen-Phelan said: "The first fix leaned on a Java default." He noted that on Mac and Linux that default already limits the file to its owner. But the team's rule demanded that permissions be set explicitly. Only the next change satisfied it.
He described the check as deterministic. He said other agents reviewing code are likely to miss problems or make them up. He also cited a Sonar survey of over 1,100 developers. By his account, 96% do not fully trust AI-generated code, yet only 48% of them fully verify it before pushing.
Why it matters
Our reading: the agent did the making in two minutes. The decision that permissions must be explicit was a person's judgment about risk, written as a rule a machine could enforce. Neither the agent's tests nor its first fix supplied it.
For managers, this suggests asking who owns the rule list. Someone must write each rule, defend it and keep it current. Reviewers also need to understand why a plausible fix still fails. That is a skill to train on purpose.
The other side
Carlsen-Phelan works for the company that sells this tool, and the evidence was one pre-recorded demo. It does not show how often rules catch what tests miss.
His own account also shows a limit. The first fix was likely safe on Mac and Linux. It failed because the team's bar was stricter than the platform default. That is a policy choice, and some teams might judge it too strict.
A rule also only catches what someone thought to write. He did not argue that judging replaces writing code. He said he likes AI review and sees the tool sitting alongside it.
Written by the WebPulse Newsroom with AI assistance, and checked by our editorial review: every quotation was verified against the recording's transcript. How we use AI.
The conversation this talking point comes from
- AI Engineer: Your AI Agent Starts Every Session Like a New Hire - Killian Carlsen-Phelan, Sonar (2026-10-30)
