Permission is not safety: an agent can be right and still break a human sign-off

Anirudh Veeraragavan, lead security engineer at Fieldguide, argued on the AI Engineer show that an AI agent can be right and still undermine a human sign-off. His example was a fictional audit. His fix is to make permission depend on the object, its state and the moment, not only on the tool.

What was said

Veeraragavan used a fictional bank reconciliation audit. An agent helps explain why a company's bank balance differs from its books. The bank showed around $2.9 million and the books around $2.7 million. The difference was explained by 41 reconciling items, which are entries that account for the gap.

A human reviewer flags one check as a possible problem. Days later, new context shows it is not an exception. That triggers the agent, which rewrites the item to no exception.

His objection was not about accuracy. The reviewer signed one version of the content. The agent then changed it. The sign-off now vouches for something the reviewer never saw. He said: "The agent didn't do anything wrong. It had the permission, it had the evidence, and it arrived at the right conclusion." Even so, he said the agent should not have changed that entry.

So he reframed authorization. The question is not only whether a tool call is allowed. The test is whether it is acceptable on this particular record, in its current state, at this moment. For rows no human has looked at, the agent may write its own draft. For reviewed rows, its output must sit beside the record as a proposal, not replace it.

Why it matters

Our reading: a tool-level permission, such as one that lets the agent edit reconciliation items, would have allowed this write. Veeraragavan's case shows why that check alone falls short. A state-aware control is simple to describe. Each record carries a status, reviewed or unreviewed. The agent writes directly only to unreviewed records. Its changes to reviewed records are stored as separate suggestions for a person to accept or reject.

This part is our argument, not the speaker's. A written policy against overwriting reviewed work depends on every person and every agent run remembering it. A status check in code does not. For buyers, a fair question to a vendor is whether the system knows which records a person has approved.

The other side

Veeraragavan's example is one fictional audit workflow, and he said the pattern applies to many other fields. The excerpts do not show how the rule works at scale. They do not say how a team should handle a reviewed item that really is wrong.

He also said the agent reached the right conclusion here. A strict no-overwrite rule therefore leaves a correct update waiting as a suggestion until a person acts. That has a cost in time. He did not say how large that cost is.