Downloadable AI model nearly matches a restricted one at simulated hacking

Restricting a tool buys time, not safety

Frontier AI labs have limited who can use their most capable hacking models. The idea was to give defenders a head start. New testing from Vals, a company that measures AI capabilities, suggests the gap had largely closed, in one narrow area, within about seven months, according to Wired. That area is automated attacks on simulated networks.

In April, Anthropic's Logan Graham told Wired that organisations should prepare for a world where these capabilities are "broadly available in 6, 12, 24 months." Wired's reading, seven months later, is that the transition "seems to have arrived on schedule." That is Wired's judgment, not a finding from the Vals data.

What Vals actually tested

Most security benchmarks use single puzzles. Vals built two cyber ranges instead. Think of a range as a replica company, with servers, workstations, user accounts and planted weaknesses. It is walled off from real systems, so an attack does no harm.

Nova, the first range, copies a big Windows environment in which several Active Directory domains trust one another. Active Directory is Microsoft's system for deciding who can log in to which machine. The second range, Lighthouse, holds two separate corporate networks. Information taken from the first is the only way into the second.

Every run begins the same way. The agent controls one workstation and knows only where a public web application lives. From there it must break in, gain administrator rights, collect passwords and move across domains. Vals scores Nova out of 21 objectives and Lighthouse out of 31.

Vals credits an objective only when the agent reads a hidden marker on a machine it truly compromised. A model's own claim of success counts for nothing. Vals says it reviewed every run and found no shortcuts.

The gap is small, and so is the price

Wired reports that OpenAI's Daybreak, available only to trusted partners, completed 81 percent of tasks on average across the two tests. The open-weight Chinese model DeepSeek completed 73 percent. Open-weight means anyone can download it.

Lighthouse was the tougher test. Vals reports that GPT-6 Sol Daybreak finished 27 of its 31 objectives in less than three hours. It entered the first company through the web application and gained administrator control. It then harvested passwords and reused them to enter the second company.

On Nova, Vals reports a tie between DeepSeek and Daybreak. DeepSeek's bill was below $3 per run. Daybreak's was around $22. Wired quotes Vals as putting the range at $17 to $46 for GPT-6 Sol and $2 to $3 for DeepSeek.

There are limits. Every model left the last objectives of the two-network range unfinished. On the single-network range, just one run in the entire set reached the final stage. These are two simulated environments, not real company networks.

Patience is cheap for a machine

One result matters more than the scores. In its first 25 million tokens, DeepSeek V4.1 Flash 1017 achieved no objectives. Most of its progress came once it passed 100 million. Vals says a test that capped compute early would have ranked it far below its real reach.

Cheap tokens made that patience affordable. Vals says the long run still cost under three dollars. Three other models, Grok 4.7, Kimi K3 and GLM 5.3 Flash, got stuck well before their budgets ran out. Vals says more compute did not help them, and every run ended on its own.

This suggests that, for these agents on these ranges, the limit was ideas rather than money or time. A slow start is no sign that a defence is holding.

The same skill works for defenders

Vals notes that open weights let defenders run these tests without waiting for a provider's approval. Rayan Krishnan, Vals's CEO, told Wired this gives defenders a strong incentive to use open models.

Wired adds that experts still debate whether AI poses a new kind of threat or amplifies old ones. This benchmark does not settle that. It does show that the cost of a long, patient intrusion attempt was low in this test.

Questions to put to your team

Start with the path the agents followed. Can anyone on your team trace a route from your public web application to your domain administrators? Has anyone tested it end to end?

Next, ask about trust relationships between domains. Vals's agents used them to move from one domain to the next. Who owns that map, and when was it last checked?

Then ask who may run an AI-driven test, and within what scope. Vals gave its agents an authorised scope and an isolated range. Your team needs the same limits before it tries this on live systems.

Finally, check how long your tests run. Short tests would have ranked the slow starter far below its real reach. By the same logic, a one-day assessment could understate what a patient agent can do. That is this article's inference, not a Vals finding.

Restriction bought defenders time. What they do with it is a management decision, not a feature of the model.

Produced by the WebPulse Newsroom with AI assistance from the original reporting credited below, and checked against that source by our editorial review. How we use AI.
Original reporting: Vals.