Levi Strauss's 51% AI throughput gain is self-reported, and output is not outcomes

Productivity figures for AI coding agents deserve a hard look. More output is not automatically better results. Jenny Patchen, a senior director at Levi Strauss & Co., raised that question about her own company's numbers. She spoke on GitHub's show, in an episode about its 17-agent software process.

What was said

Patchen described specialised AI agents. They handle steps from requirements to deployment. The show notes say five to eleven independent agents check each pull request, meaning a proposed code change. Humans keep the final approval.

She then gave results. Throughput rose 51%, she said. The team had expected it to fall at first, because the change was so big. She also said the share of AI-assisted pull requests grew from 26% to 69% over a short window.

Then she raised the obvious question. Throughput is rising, but are agents just creating more? Productivity, she said, does not necessarily track the outcomes you want. Her answer was that the company released its product much faster than the legacy version.

She also described how agents fail. In her words, "It's very common for agents to run ahead with total confidence and then end up giving you something that didn't match your intent."

Why it matters

Our reading: these are the company's own numbers, shown in a conference session. They show adoption and speed. They do not show that the code is better.

The share of AI-assisted pull requests measures how much work agents touch. It does not measure how well that work holds up. If agents produce changes faster, checking may become the limit. That is our argument, not hers. Patchen did say "you can only go as fast as you have good brakes."

Managers and buyers should ask more than how much faster. Ask what checks grew along with the output. Ask who still reads the work. Levi's answer was many reviewing agents plus human approval.

The other side

Patchen's account is not a warning against agents. She called the results great. She said the team wants AI-assisted pull requests to reach 100%. The faster product release is a real outcome, not just a count.

She also cited a pass-rate figure as a sign that quality improved. The transcript number is unclear, so we leave it out. The excerpts do not say how throughput was measured. They also do not give the team size or the length of the window.

Her account shows limits too. Agents working in parallel caused merge conflicts, where changes clash. The team changed its approach more than once. The conversation did not settle whether review can keep up as output grows.