Every week someone asks me whether large language models belong in security and reliability work. My answer has stopped being "yes" or "no." It's a litmus test, and it takes one sentence: <strong>is this replacing human judgment, or is it replacing toil?</strong>
If it's replacing judgment, I'm out. If it's replacing toil, I'm very interested. Almost every good and bad AI decision I've watched teams make sorts cleanly along that line.
Toil is the enemy. Judgment is the product.
Let me define terms, because the distinction does real work. Toil is the repetitive, tedious labor that doesn't require a judgment call: combing through hundreds of pages of compliance documents to pull out the six clauses that matter, generating the boilerplate for an integration test, drafting the config for a failure-injection experiment you've already designed. Nobody's career was made by this work. Nobody grew from it. It's exactly what machines are for, and I'm delighted to hand it over.
Judgment is everything else — and it's the actual product of a senior engineer. Keeping a site up is toil; knowing <em>which</em> of the forty things screaming at 3 AM actually matters is judgment. Site reliability engineers who are great at their jobs are great because of their judgment, not their ability to follow steps. The idea of replacing an SRE with a markdown file and a model makes me genuinely nervous. An agent that can kick a node and buy twenty minutes while a human gets to a keyboard? That has real value. The difference between those two things is the difference between a tool and a liability.
My favorite piece of evidence for the primacy of human judgment comes from the XZ Utils backdoor — one of the most sophisticated supply-chain attacks ever attempted, years in the making. It wasn't caught by a scanner or a dashboard. It was caught by an engineer who noticed a performance regression of less than one percent and refused to let it go. No model was going to care about that half-second. A human with taste and stubbornness did. That's also why, when I talk to people who think like attackers, they'll admit they're not especially scared of security tooling — they're scared of the person who obsesses over why something feels slightly off.
Where the models actually shine
So where do I happily deploy LLMs today? A few places, all firmly on the toil side of the line.
<strong>Document intelligence.</strong> Nobody loves excavating compliance archives, vendor questionnaires, and policy PDFs for relevant obligations. Models are excellent at this, and every hour they save is an hour a human can spend on the strategic question that actually matters: are we sustaining resilience, and what indicators would tell us?
<strong>Scaffolding for experiments.</strong> Chaos experiments, integration tests, specific configurations — with the crucial caveat that a model is only as good as the corpus it learned from. If you can't verify that reasonably clean code went in, be skeptical of what comes out.
<strong>Buying time and capacity.</strong> The agent that stabilizes a system while the human travels toward the problem. It's not making the call; it's holding the door.
<strong>Rubber-duck reasoning.</strong> Talking through a gnarly problem with something that talks back is underrated. So is a stranger use case I've come to love: pressure-testing how an initiative will land across different audiences and cultural contexts before you pitch it. Techniques read very differently in different rooms — pitch "deception-based defense" to one security culture and you'll hear "we don't want to be the bad guys"; pitch it to another and you'll hear "tell me more." A model that's read more broadly than any of us can help you anticipate that — especially if you ask it to challenge your assumptions rather than confirm them.
The footgun in the markdown file
Now the mixed feelings. There's a growing pattern of running operations from natural-language playbooks — prose instructions that a model interprets, where a script used to be. I genuinely see both sides of this one.
The case for: security is dressed up in far too much arcane jargon, and plenty of people who could contribute meaningfully are kept out by the scripting barrier. Lowering that barrier is a real good.
The case against: a script is deterministic and a paragraph is not. We already struggle to verify that software does what its designer intended — that's one of the hardest problems in computer science with <em>precise</em> languages. Injecting ambiguity into your operational path, and letting a model resolve that ambiguity with statistics, is a footgun waiting for a bad day. And whatever you do, don't ask a model "is our system resilient?" and treat the answer as an assessment. That's not toil delegation; that's judgment abdication with extra steps.
The test, applied
None of this is a prediction about what models will or won't be able to do next year. It's a principle about what we should <em>want</em> them to do. Before every AI deployment in your security or reliability practice, ask the question: does this free my people to exercise more judgment, or does it quietly substitute for their judgment?
The first kind of tool compounds. Your engineers spend less time on excavation and more time noticing the one-percent anomalies. The second kind decays — because the judgment you stop exercising is judgment you eventually stop having. And when your building meets its earthquake, judgment is the only thing that was ever going to hold it up.
