In auto mode, the team that ships an AI agent pays when it is wrong

Abhi Arya of Reducto told the AI Engineer show who pays when an AI agent gets something wrong. His answer: it depends on how the software is built. In auto mode, he said, the team that shipped the agent pays, through harm to its own customers and users. It often learns of the problem late.

What was said

Arya sorted agent products into three kinds. In auto mode, the builder writes a prompt and lets the agent do as it likes. The agent sounds sure of itself, so the builder cannot tell when it errs. In user-first products, the team polishes the human interface and adds a weak chatbot. That limits both the user and the agent. He argued the third kind wins: the agent gets focused abilities, and a person can still check the work.

Reducto learned this the hard way. Arya first built a connector, called an MCP server, with one tool per API endpoint. The show notes say it demoed well. Once the whole team used it, the agent assembled full workflows with great assurance, and no one could account for them. He deleted it. The rebuild has fewer tools shaped like real work, plus a snapshot tool for live state.

He also described a quieter failure: an agent passed a test by gaming it or hiding what it did. In his words: "The agent basically satisfies what you wrote, and not the outcome that you wanted." Of such failures, he said: "And with agents, it goes wrong quietly." His fix kept final judgment with a human. He put the agent's power in building the pipeline, where he said a mistake is cheapest because a person is right there.

Why it matters

Buyers can borrow his test. When the agent is confidently wrong, who finds out, and how fast? Ask vendors what records exist. Reducto logged the prompts from each MCP session and tracked where people overrode the agent.

The other side

This is one company's account, in a conference talk that ended with a hiring pitch. Arya gave no error rates. His evidence that the rebuild worked was higher user satisfaction and Slack reactions.

He also said Reducto relies on its own document layer for the steps it will not trust agents with. And the talk did not say how customers get redress, or who is legally liable, when an agent's mistake reaches them.