- A Langfuse banking demo using fictional data counted 89 interactions in a week where the agent may have given investment advice; it is not a real bank's figure.
- Separately, a speaker said regulated firms should write hard limits, like never giving investment advice, into the evaluation set used to score an AI agent.
A workshop on the AI Engineer show, led by Doneyli De Jesus and Annabell Schäfer of Langfuse, made two separate points about AI in banking. First, a demo dashboard on fictional data counted 89 interactions flagged as possible investment advice in one week. That figure is illustrative, not a real bank's rate. Second, a speaker urged regulated firms to write hard limits into the evaluation set. That is the collection of test cases used to score an AI agent.
What was said
The demo used a fictional banking assistant. Each customer interaction carried a categorical score, meaning a label. One label recorded the intent of a security risk, such as investment advice. The dashboard counted those labels. The presenter could then filter down to the flagged cases and read them.
The count was 89 over seven days. In those interactions, a customer sought investment guidance and the agent may have supplied it, or attempted to. The presenter noted that a customer-facing agent carries liability, financial and reputational risk.
Near the end, a speaker on the show gave the second point. A speaker said earlier that nothing like a unit test exists for AI applications, so teams need another way to check them. For regulated work, the advice was to define hard limits: "For financial services, for example, it should never give investment advice." The speaker wanted that built into the evaluation set. The modest aim was that the behaviour should ideally never happen, but "at least you will know about it when it's going to happen."
Why it matters
Our reading: the two points fit together, though the speakers did not link them. A hard limit written as a test case turns a vague promise into something a team can check after every change. A label on customer interactions, as in the demo, would show how often the limit is being approached.
Ordinary software has unit tests, which are small automatic checks. AI agents lack that, so a banned behaviour can go unseen. For anyone buying an agent in a regulated field, a fair vendor question is how it counts breaches of each hard limit. The cost of a breach falls first on the customer who acts on bad advice, then on the firm.
The other side
The excerpts never say the 89 came from a rule in the evaluation set. They show a dashboard label on the demo's fictional interactions. The wording was also cautious: the agent "might have" given advice, so these are flagged cases, not confirmed breaches.
The approach detects rather than prevents, and the speaker's stated goal was to know when it happens. The excerpts do not say how accurate the flagging is, who reviews the 89 cases, or what happens to a customer who already got the advice. The workshop also comes from a company that sells evaluation tooling.
Written by the WebPulse Newsroom with AI assistance, and checked by our editorial review: every quotation was verified against the recording's transcript. How we use AI.
The conversation this talking point comes from
- AI Engineer: Evals for High-Stakes AI Agents — Doneyli De Jesus & Annabell Schäfer, Langfuse (2026-10-18)
