An agent that warns but cannot stop the work leaves the cost to others

An AI agent that spots a problem but cannot halt the work has not made anything safe. That was the point Stephanie Jarmak made on the AI Engineer show. She builds evals, meaning tests for AI systems, at Omni Analytics. Her argument: stopping must be enforced by code, and a human should be brought in when needed.

What was said

Jarmak described a software factory she is experimenting with. Agents do work behind approval queues and required checks, and she runs regular audits. In one case the agent noticed things going wrong. But, she said, "there was no mechanism in place for it to actually like enforce a stopping situation."

Her fix is to separate two jobs. One is flagging a problem. The other is a gate in the code that blocks the work. Only then do you decide how a case reaches a person.

She also warned about overloading reviewers. She described research on an AI judge, a system that grades other AI output, with a 1% false alarm rate. If every alarm went to a human, she said, that would mean 10,000 reviews. She asked whether an organization has the experts for that.

Records can mislead too. In a study she cited, researchers planted security issues. A tamper check using hashes passed everything. An independent comparison of traffic history and model changes caught all of them. Her stance: "I want to automate as much as possible for the work to get done, but I also do not trust these systems, like, at all."

Why it matters

Our reading: a warning that nothing acts on moves the cost to someone else. The work carries on, and the people downstream absorb the result. Those could be customers, the next team or a small business relying on the output.

For buyers and managers, the useful question is what stops the work, not what notices a problem. Ask which checks are enforced in code and who receives an escalation. Ask whether the record of events can be checked independently.

Jarmak said she sets boundaries for when an agent may deploy on its own and when it must stop and escalate. That is a design decision someone has to own.

The other side

Gates are not free. Jarmak was clear that human review is a bottleneck. The excerpt offered no fix beyond tuning alarm rates and weighing how consequential each alarm is downstream.

Her main example came from her own experimental setup, not a production study. The 1% alarm rate and the tamper-check case came from other research she described, with few details. She also said each team must decide how much risk it will accept, so there is no single right threshold.