AlixPartners guests describe AI errors that raised no alarm in contract analysis

Natalia Connolly and Kevin Madura of AlixPartners spoke on the AI Engineer show about a test of Claude Code on real contract data. Its first answers looked fine but held two errors. The guests described several failures that raised no visible error, and one of them hid $800,000.

What was said

The guests ran Claude Code on a procurement review after a company merger. It covered hundreds of vendors and over 40,000 contracts. One total was wrong by about $800,000, and a key renewal date was a month late. A speaker on the show said the answers were polished: "They look complete. They're certainly well formatted. They look plausible enough, you know."

The $800,000 miss came from an orphan amendment. That is a change to a contract that was never linked to its parent contract, so Claude Code could not find it. The excerpts do not say what caused the late renewal date.

The guests also listed other risks. Very long contracts can hang during processing. If the code catches the timeout and moves on, a speaker said: "You never even know that the contract never made it into the database." They gave this as something that can go wrong, not as the cause of a specific error.

On access, they said a few calls returned a malformed value or a zero, or hit a permission block. Claude Code got no useful feedback and went ahead, apparently treating an invoice as zero. Their lesson was to tell the agent what failed, so it can recover or alert a person.

Why it matters

Our reading: silent failure is where AI makes an old weakness larger. A gap in the data used to touch one analyst's spreadsheet. Now it can feed every answer an agent writes, overnight, in a clean format that invites trust.

For people who buy or run these systems, the questions are simple. When a step fails or a file is skipped, does the system say so? Does a zero mean zero, or "could not read"? Can each figure be traced to its source record? The guests built a five-stage pipeline so that answers could be traced.

The other side

The guests were open that this took effort. They spent about two weeks building the data pipeline before the real analysis began. They also said that real contract portfolios hold problems that may not show at the start. A pipeline built in advance covers only the problems someone has already thought of.

One problem stayed open. The agent suggested savings that management had already begun. Getting the right business context into the agent is, in their words, a tricky problem. The excerpts do not say how many contracts failed silently.