- Eli Cohen of Snyk argued that a pen test once or twice a year leaves most of the year untested while code changes constantly.
- The cost of keeping the ritual is the gap between tests and the slow path from report to fix, not only the fee.
Eli Cohen of Snyk argued on the AI Engineer show that the yearly or twice-yearly penetration test no longer fits how software is built or attacked. A pen test hires a human to try to break into your application. Cohen said keeping that rhythm leaves roughly 350 days a year with no testing at all.
What was said
Cohen called the pen test the gold standard and the best method for testing business logic, meaning the rules a product follows. But a human researcher is expensive and slow, so companies run it one to three times a year. The output is clumsy too, he said: a long PDF that developers must sort into real and not real, fix, then retest. "It sounds like a movie from the 90s, right?" he said.
Meanwhile, he said, attackers use AI around the clock. He claimed the average successful AI attack takes 34 minutes and the fastest takes four. His challenge: "what are you going to do with the rest of the 350 days you have a year that no one tested your application?"
Cohen described the alternative as continuous offensive testing. One piece is agent red teaming: multi-step attacks that simulate prompt injection (tricking an AI with planted instructions) and data exfiltration (stealing data). Another is dynamic testing, which probes a running app and is good at finding authorization flaws, meaning who may see what. A third is LLMs that reason about application context, which he said is where new vulnerabilities come from. The last is proof: showing a bug is exploitable, not merely flagging it.
Why it matters
Our reading: the price of the ritual is the gap between how often software changes and how often anyone attacks it on purpose. Cohen said code now reaches production nonstop, while tests follow the calendar.
A budget signer can act on that. Match testing cadence to release cadence. Scope the human pen test to business logic, where Cohen said humans do best. Ask of any report whether findings are proven exploitable. Track the days from finding to fix, since the report-to-retest loop he described has many handoffs.
The other side
Cohen sells a product here. His Evo offering is tied to Snyk, though he called his principles general and told listeners to check them with any vendor. The excerpts give no source for the 34-minute and four-minute figures, so treat them as his claims.
Cohen said the approach would be cheaper, but gave no figures in the excerpts. They also show no evidence that AI testing matches a human on business logic, and they do not explain why that is hard to automate. He spoke of combining tools with pen testing, so he did not call for dropping it.
Written by the WebPulse Newsroom with AI assistance, and checked by our editorial review: every quotation was verified against the recording's transcript. How we use AI.
The conversation this talking point comes from
- AI Engineer: AI Hackers Are Faster Than Your Pen Test — Eli Cohen, Snyk (2026-10-07)
