- NIST's CAISI found Z.ai's open-weight GLM-5.3 trails the U.S. frontier in cyber tasks by about four months. No tested open-weight model scores higher.
- Anthropic separately alleges Zhipu distilled U.S. models' cyber skills. It says new defenses cut data harvested by distillers by about 90 to 99 percent, on early evidence.
- Ask your security team how long it takes to patch after a bug is disclosed, and ask AI vendors how they handle distillation.
A copied skill does not bring its safeguards along
A safeguard belongs to a service, not to the skill the service performs. If someone copies the skill by studying the service's answers, the guardrails stay behind. Two documents published weeks apart bear on that idea, and it matters to anyone who owns software that attackers might probe.
What NIST measured
NIST's Center for AI Standards and Innovation (CAISI) assessed GLM-5.3, a model from the China-based company Z.ai, formerly Zhipu AI. Z.ai released it on August 14, 2026. It published the model's weights, the files that let anyone run it, two weeks later.
CAISI tested the model on four benchmarks for finding and exploiting software flaws. No open-weight model it has evaluated scores higher. CAISI also concluded that GLM-5.3 sits clearly below the best released U.S. models, by margins well outside its 95% confidence intervals. The aggregate gap is about four months. GLM-5.3 does score above Kimi K3, the previous leader among PRC models CAISI had tested.
How the test works
The benchmarks are demanding. The first set has 183 tasks. Each hands the model the code of a browser engine, V8 or SpiderMonkey, which sit behind Chrome, Firefox and Edge. The code holds a known vulnerability, and the model is told which files to audit. It must locate the weakness and write code that crashes the engine by triggering it.
A second set of 41 tasks asks the model to turn a known V8 bug into an exploit that runs code of its choosing. A third set of 502 tasks does the same for real open source bugs. A fourth set of 297 tasks gives no description at all. The model must find the defect and use it to hijack the program.
CAISI rolls the results into a "cyber capability index" borrowed from exam-scoring statistics. Every 400 points on the index equal a tenfold change in the odds of solving a task.
The limits are stated plainly. CAISI did not compare against models that exist but are not yet released. Those could be stronger. U.S. models were also tested with their cyber safeguards disabled, where applicable. So the comparison shows raw capability, not what a typical user gets from a guarded product.
Anthropic's allegation: distillation
CAISI does not say how Z.ai built the model. Anthropic makes a separate claim, in a post published October 12. It says Zhipu distilled the cyber capabilities of leading U.S. models while developing GLM-5.3. Anthropic does not say this explains where the model sits on CAISI's index. Distillation means collecting millions of answers from a stronger model and using them to teach a smaller one to imitate it. Anthropic says this is often done through networks of fraudulent accounts.
Anthropic says Zhipu first went after Claude Fable 5, but that model's cyber protections weakened the results Zhipu could harvest. In Anthropic's account, Zhipu then turned to an older Claude model and to a model from another lab. These are Anthropic's claims. The source material contains no response from Z.ai.
Anthropic also reports its own research on the model. It says GLM-5.3's protections are easy to get around, and that the model will help build working exploits from start to finish. The NIST assessment excerpt does not address that finding.
Why reasoning is the target
Anthropic says the most valuable data for distillers is a model's written-out reasoning. It shows how the model reaches an answer, not just the answer. Its defenses therefore focus almost entirely on making that reasoning harder to reach.
Anthropic calls this early evidence, and it is the company's own measurement. It also reports costs. The company says it gave up significant revenue, including paying traffic from distillers worth trillions of tokens. Some changes forced legitimate customers to update their integrations.
What leaders should ask
First, ask your security team how long it takes from a public bug disclosure to a deployed fix, for browsers and open source dependencies alike. CAISI's exploit-development tasks test a close cousin of this step: turning a known bug into a working exploit. A four-month gap to the U.S. frontier is a measurement against released models only. It is not a reason to relax.
Second, if you sell an AI product or API, ask who can read your model's reasoning output, and how you would spot coordinated fraudulent accounts. Third, ask your AI vendors how they handle distillation, and which integration changes they may require.
The safeguard is part of the service, so the skill can be copied without it. Plan for attackers who have the skill but not the guardrail.
Produced by the WebPulse Newsroom with AI assistance from the original reporting credited below, and checked against that source by our editorial review. How we use AI.
Original reporting: NIST Center for AI Standards and Innovation (CAISI).
