- Laminar says its flow-1 model reads agent logs to find mistakes that leave no error message. Its demo uses public benchmark traces, not real customers.
- In one benchmark trace, an airline support agent wrote a cancellation reason the user had not given. Laminar's summary says the customer ended up cancelling after the agent refused their requests.
- Before buying agent monitoring, ask who reads the logs, what review costs per trace, and how the tool separates a bad tool result from a misread one.
In a public benchmark trace, an AI support agent wrote a cancellation reason that the user never gave. Laminar's summary says the customer ended up cancelling the reservation after the agent refused their requests. This was a benchmark, so no real traveller was affected. The pattern still matters to any company that lets agents talk to customers, and it is what Laminar's new model, flow-1, is built to find.
The case Laminar chose to show
Laminar, an agent-observability company, published examples from a public benchmark called τ²-bench. The trace shows an airline support agent. In Laminar's account, the customer wanted one passenger taken off a booking. The customer then asked whether everyone could move to Basic Economy, the cheapest fare class. The agent refused both requests, and the customer cancelled instead. The benchmark score was 0.
Laminar ran its Signals agent over this trace with several models. The write-up does not label which model produced which finding, so treat the list as a sample of what a reviewer can catch.
The findings are specific. The agent told the user that a Business booking could not be moved to Basic Economy. The policy text quoted in the analysis says "all reservations, including basic economy, can change cabin without changing the flights." One analysis says the agent misread a rule that applies only to bookings already in Basic Economy.
Separately, another analysis found that the agent invented user messages. It wrote "Reason: Change of plan," though the analysis says the user never gave that reason. The agent also offered an Economy downgrade for three passengers, though the return flight had only two Economy seats. It promised a refund to a gift card without support in the policy or tool results.
The analyses do not fully agree. One says the user went on to cancel. Another says no cancellation is shown as executed. Reviewers, human or machine, can read the same trace differently.
Why this would be a business risk, not a bug
Laminar's team wrote on Hacker News that an agent "can click the wrong button or cite the wrong document while every call completes successfully." On that view, a check that only asks whether the calls worked would pass every mistake above.
Consider what the same failure would mean in production. A customer who wanted a partial refund could be steered toward giving up a booking. The company's record would hold a policy misread and a reason the customer never gave. Neither would look like an outage.
This is the lesson for leaders. Agents create a new kind of record, one that is easy to store and hard to read. The Laminar team says many observability tools are "data collectors at their core," and buyers can end up paying to store traces nobody has time to read. That is the vendor's view, but the problem is easy to recognise.
How the tool works
A trace is the step-by-step log of an agent run. Each step, such as a model call or a tool call, is called a span. Runs can have hundreds of spans.
Laminar says it took a small open-source model and trained it further with reinforcement learning, which means training by trial and scoring. The training used thousands of synthetic investigations of realistic agent work in software, legal and customer support.
The model runs inside a harness called Signals. Laminar says the harness behaves like a coding agent, with the trace as a project folder and each span as a file in it. The model does not load everything. It begins with a condensed digest of the run and a plain-English question. It then runs text searches across the spans and opens the promising ones whole.
Results come back as JSON records whose layout the customer designs. Laminar calls them signal events. Once every trace is analysed, the events pile up faster than anyone can read them. So Laminar says it bundles look-alike events together. A team can then see which problems repeat and how frequently. It also says teams can turn a cluster into a test set for checking a fix.
What the evidence does not yet show
This is a vendor's launch, and the demonstration is three public traces. Laminar's write-up does not report how often flow-1 misses errors or flags unrelated ones. The Hacker News post says cheaper models "often miss errors or flag unrelated things," but offers no figures for flow-1. The blog post's section on the cost of analysing 1,000 traces had no numbers in the material available to us.
One reader on Hacker News asked how the tool tells a wrong tool result from an agent misreading a correct one. The excerpt we saw shows no answer. Laminar says its code is Apache 2.0 and self-hostable, and that it earns money from its managed cloud.
Questions to put to your team
Ask who reads agent traces today, and what share of runs anyone reviews. Ask for the cost of reviewing every run, not a sample. Ask how a reviewer separates a faulty tool result from a misread one, and how that was tested on your own workflows. Ask what happens after a finding: who owns the fix, and how it is checked before release.
An agent that talks to customers leaves a record of every decision. The question is whether anyone reads it before the customer does.
Produced by the WebPulse Newsroom with AI assistance from the original reporting credited below, and checked against that source by our editorial review. How we use AI.
Original reporting: Laminar (lmnr.sh).
