- Microsoft researchers report that editing fewer than 10 dormant neurons, with no training, can backdoor an LLM-based agent while it passes standard evaluations.
- Reliable detection needs a comparison with the original weights, so Microsoft says knowing where a model came from matters most.
- Leaders should ask where each deployed model came from, what access its agent holds, and whether anyone watches what it runs.
A model can pass every test and still carry a switch
Most companies judge an AI model by how it behaves in testing. Microsoft Research has published work that shows the limit of that habit. Its researchers describe a technique called Lazarus. It edits a model so that it behaves normally until a hidden trigger appears.
The researcher who found it says the edited model "passes standard evals and behaves normally, until a trigger appears." The lesson here is plain. Good behaviour in a test tells you how the model acted in that test. It does not tell you whether someone has changed the model since.
What Microsoft's researchers found
Microsoft's researcher reports that rewriting the weights of fewer than ten rarely used neurons can plant a backdoor in the model behind an agent. No training step is involved. Dormant neurons are parts of the model that seldom activate in normal use.
The example trigger is a calendar date. When the clock rolls over into 2027, an agent working on a routine job could output a command that fetches and runs the attacker's code. The agent would run it with whatever permissions it holds.
The study is described in a Microsoft blog post and a linked paper. This story draws on the blog summary only.
How the trick works
According to the researchers, large language models contain many neurons that seldom activate. Each one can be rewired so that it activates only when a trigger is present. Pretraining arranges a model's internal representations so cleanly, they say, that a basic linear classifier trained on roughly 50 examples can separate triggered inputs from ordinary ones.
Once a rewired neuron activates, it can force the model to output a chosen token, the small unit of text a model produces. Chaining several neurons builds longer payloads, including commands that run in a terminal.
The compute cost is low. The researchers say the attack needs only the hardware used to run a model, so a machine that can load a model can also alter it. The attacker needs write access to a copy of the weights, the file that holds what the model learned.
Why agents raise the stakes
A chatbot that emits a bad command produces text. An agent with terminal access runs it. Microsoft calls the threat especially serious for agents with tool or terminal access. It lists three outcomes: downloading and running attacker code, taking secrets, and running destructive commands.
The researchers also say this is not a case against open-weight models. Internally built models are equally exposed. A compromised account, pipeline or storage system can be enough. The question is whether the weights you run are the weights the developer released.
What the defence looks like
There is some good news. The researchers say frontier agents can spot anomalous weight edits if asked to look. They add that no scan can prove beyond doubt that a model is safe. Reliable detection requires comparing against the original, unmodified weights.
That makes provenance the main control. Think of it as a paper trail: who published the model, and proof that the file you run is the file they shipped. The researchers also describe a backdoored model as compromised software. Their advice is to treat models as possible insiders: extend limited trust, narrow their permissions, and watch their activity.
Microsoft describes its own controls for Foundry, its model platform. It says Foundry ties open-weight models to their original publisher and runs them in Microsoft's own containers. In its words, these controls address a model's origin and the software environment around it. These are the company's statements about its own product, and this report did not test them.
Microsoft also names a remaining risk. A trusted provider could itself be compromised, through its training systems or an insider. The company says it is working on a red-teaming harness and on weight scanning with frontier agents. It says signed models with tamper-proof audit trails are being planned.
Questions to put to your team
Ask where every model in production came from, and whether you can show it is unchanged since release. Ask who holds write access to the stored weights. Ask what each agent is allowed to run, and whether anything raises a flag when an agent fetches and runs code it has not used before.
Ask also whether your vendors sign their models and keep audit trails. The researchers say scanning weights is not yet common practice and should change.
One more detail deserves a pause. The work began as a quick experiment in March. The researcher asked an AI coding assistant, Claude Code with Opus 4.5, to hardwire a few activations into a model without training, and it worked. That became a months-long study. The post says the assistant found most of the method on its own. Microsoft presents this as a pattern to encourage: agents surface problems, then people study and fix them.
The practical takeaway is simple. An AI model is software, and software can be altered. Treat it like any code from outside your company, and verify it before you grant it access.
Produced by the WebPulse Newsroom with AI assistance from the original reporting credited below, and checked against that source by our editorial review. How we use AI.
Original reporting: Microsoft Research.
