One failing compliance score should block an AI release, whatever the average says

Doneyli De Jesus leads AI engineering solutions at Langfuse, which is part of ClickHouse. On the AI Engineer show he argued that one failing score should stop an AI release, even when the average of all scores looks healthy.

What was said

De Jesus described work with a top-five bank, according to the show notes. His team scored an AI agent across several dimensions, such as groundedness, completeness, tool use and regulatory compliance. A dimension is simply one thing being measured.

The agent did well on the first three. On regulatory compliance it scored zero. De Jesus said that score is binary: it is either zero or 100. A bank cannot ship something that breaks a regulatory requirement, he said, because of possible penalties and reputational damage.

He rejected the idea of averaging the scores and passing anything above 90%. In his words, "it might be that one dimension actually completely shuts down the gate for you to move this into production."

He also said the bar differs by use case, and that risk people should help set it. Failed runs go into an annotation queue, a list of cases for domain experts to review. He summed up the approach this way: "you have to gate the application, not the model." His team said this cut evaluation from weeks to minutes.

Why it matters

Our reading: the skill here is not building the agent. It is deciding which checks are pass-or-fail, where the thresholds sit, and who defends them when a release is late.

An average lets strong scores hide a fatal one. A manager who signs off on a dashboard of averages may be approving the exact failure the gate exists to catch.

The practical questions for buyers and builders are simple. Which dimensions can block a release on their own? Who set each threshold? Who reads the failures? De Jesus said that work needs domain experts, not only technical staff.

That points to training. People who review failed traces need to know the rules of their field well enough to say what a failure means.

The other side

This is one team's account of one bank. Compliance suits a binary gate because a rule is either broken or not. The excerpts do not say how the compliance check itself was scored.

A gate is also only as good as the golden data set behind it, the set of example questions and expected answers. De Jesus described a loop in which expert-reviewed failures can improve that set, but he did not claim it is ever finished.

He said plainly that there is no template for this. The speed gain from weeks to minutes comes from his own team, not an independent test.