An AI agent can obey every rule and still do the job badly

Tim Tyler of Gravitee argued on the AI Engineer show that an AI agent staying inside its permissions does not mean it is doing its job well. He said an agent's oversight scorecard needs two kinds of check. One covers behaving, with firm rules. The other covers succeeding, with judged intents.

What was said

Tyler presented what he called the Agent Accountability framework, which treats agents as a digital workforce. In its last stage, administration, companies oversee the agents. He said the usual questions are about scope: is the agent out of bounds, or touching things it should not? Passing those checks is not enough. "That does not always equal succeeding," he said.

He tied behaving to an agent's authority and succeeding to its role. Authority checks are firm rules. His example was an invoice agent approved only up to a set dollar limit. It either stays under the limit or it does not.

Role checks are about intent. Did the agent match the correct purchase order and the right vendor? Is its invoice match accuracy where the company wants it? Is the time it takes to reconcile within range? Tyler admitted you can put rules on some of these. But he said they are better treated as a scorecard, with weights drawn from the role and authority definitions. His summary: "rules are checked, intents are judged." He said an intent should be judged against a confidence level.

Confidence also decides the response. If the company is fairly confident, it takes one action. If confidence is low, it takes another. He compared this to how people are handled: a correction first, then narrower authority, then escalation, then removal.

Why it matters

Our reading: this moves part of the work from making to judging. A manager who buys or runs agents cannot sign off on a clean permissions report alone. Someone must say what a good result looks like for each role, such as matching the right purchase order. That person must then read the confidence scores and decide what happens at each level.

That is a job for people who know the business process, not only for security staff. Tyler's weights also come from the role definition. So whoever writes the role is shaping how the agent is graded.

The other side

Tyler himself blurred the line. He said you can put rules on intent-style measures such as accuracy and cycle time. The two kinds of check are not cleanly separate.

The excerpts also leave questions open. They do not say how a confidence level is worked out, or who checks the software guards that do the judging. Tyler called these "guardian agents" and said they work at machine speed. They are still software, and the talk, as excerpted, does not say how they are held to account.

Nor do the excerpts show results from any company using this approach. It is a framework one speaker presented, not evidence that it works.