Langfuse says expert judgment should set the tests every AI change must pass

Langfuse argues the scarce skill in AI engineering is shifting from building to judging. Doneyli De Jesus and Annabell Schäfer of Langfuse made that case in a hands-on workshop on the AI Engineer show. They used a banking assistant to show the loop. An expert gives raw opinions for about an hour, an LLM turns them into named failure modes, and those become checks that every change must pass.

What was said

The presenters start from traces, the logged record of what an agent actually did. Nobody can read thousands of them, so a team pulls a representative sample, such as conversations about disputes. It goes into a queue for a domain expert, like a risk specialist.

The speakers said to keep the expert's job easy. Give a simple score and a short comment, perhaps dictated. They suggested asking for an hour of time.

The expert does not invent categories. An LLM reads all the comments and groups them into failure modes. As a speaker on the show put it, LLMs "are great at pattern matching. They are great at extracting information from text." Those failure modes then become the quality measures that each test run reports. The show notes add that the pipeline can block a prompt change when scores drop.

Why it matters

Our reading: the expensive input moves from writing tests to supplying judgment. For managers, the bottleneck becomes access to a few busy experts and a clean way to capture what they know. The workshop gave a concrete example in regulated work: a financial assistant should never give investment advice, and that limit belongs in the evaluation set.

It also affects training. One presenter said the other once spent days reading traces as a Langfuse user. The workshop said that reading still matters for understanding agent behaviour. But it said instrumenting and sifting for failure modes can now be automated. Teams will need to decide how newcomers build the judgment that the reading used to teach.

The other side

The speakers work for Langfuse, which sells tooling for this loop, and the workshop was a demonstration with fictional data. It was not an independent test.

They also admitted limits. LLM judges can be wrong, so they advised mixing them with fixed rule-based checks. They said the judge itself must be checked against human labels. The practical route, they said, is one person labelling a sample, then rechecking over time. They suggested starting with too many false positives and tuning down. Nothing in the excerpts explains how to grow the "taste" they say humans must bring.