Ramp says a swarm of 10,000 AI agents found a flaw no single component showed

A chain of parts that are each safe can still fail when joined. Alexander Shevchenko of Ramp Labs described such a case on the AI Engineer show. He said a swarm of 10,000 AI agents, run against Ramp's own systems, found a flaw that no single component showed.

What was said

Shevchenko said Ramp used 10,000 parallel agents to pen-test the whole company. A pen test is an authorised attack meant to find weak spots. A cloud agent called Ramp Inspect handled the setup. The agents hunted for vulnerabilities and filed tickets. Further AI models then merged duplicate tickets and tried to reproduce each flaw as an attacker would, to check it was real and important.

He said the result was seven high-severity vulnerabilities, all patched. One was a script injection that could arrive by email. In his words, "no single component of that stack was unsafe, and the kind of swarm had to patch together multiple components". He said the swarm needed to understand several components end to end.

He also said the run is automatic. "You don't need to spend anyone's time," he said. On cost, he said Ramp's write-up found that open-weight models, which are models others can run themselves, can be up to five times cheaper than frontier ones. A swarm can spend more on those cheap tokens, the units a model reads and writes, and still get almost as good a detection rate.

Why it matters

Our reading: a review that checks one component at a time can miss a flaw that exists only across several. A swarm that reads the stack end to end looks for exactly that.

If finding candidates costs little, the scarce skill may become deciding which findings are real and worth fixing. Shevchenko described Ramp handing that step to more AI models, so this thesis goes beyond what he said.

For managers who buy or run software, a useful question for any security test is whether it looks at chains of parts, not only the parts. Cheaper models are open to anyone, including attackers. That is our inference, not something Shevchenko said.

The other side

This is one company's account of its own test, given in a conference talk. The talk gives no count of false alarms, no total cost and no comparison with a human-led pen test. Cheaper models caught almost as much, not all, so some findings may be missed.

The checking step also uses AI models. The talk does not say how often they wrongly dismiss a real flaw, or how people fit into the process. Seven patched issues is a useful result, but it does not show how well the method works elsewhere.