Skip to content
Business Efficiency Talking point

Jellyfish: cost per merged PR fell 26%, but it doesn't capture review effort

Cost per merged PR fell 26% in Jellyfish data; its speaker flagged review stress but did not measure it.

W
WebPulse Newsroom
AI-assisted · 2 min read
Share on X LinkedIn
Jellyfish: cost per merged PR fell 26%, but it doesn't capture review effort
In brief
  • Jellyfish's Steve Jamison reported that cost per merged pull request fell 26% as AI spending rose, mainly because salary is spread across more PRs.
  • The same data shows PRs are not smaller, and he tied that to review stress. The cost metric does not measure that strain.

Steve Jamison of Jellyfish argued on the AI Engineer show that teams spending heavily on AI are shipping cheaper pull requests. A pull request (PR) is a proposed code change that a colleague reviews before it merges. The saving comes mostly from spreading salary across more PRs. His data also shows those PRs are not shrinking, and he linked that to growing review stress.

What was said

Jamison drew on Jellyfish engineering data. He reported a 60-fold rise in AI spending and 43% more merged PRs per engineer. Combined cost per merged PR fell 26% over the same period.

His explanation is that salary dominates the sum. He put AI at roughly 2.5% of what each additional merged change costs, with salary covering the other 97.5%. When the same salaries produce more PRs, each PR carries less of that cost.

He then tested whether the extra throughput came from smaller PRs. It did not. He said the gain did not come from smaller changes, and that lines of code per PR are rising. He offered reasons: generative AI tends to produce a lot of code, and developers fold extra edge cases into one PR to have fewer to manage.

He tied this to a known pain point. In his words, it "doesn't mesh well with the PR review stress folks are feeling these days though."

Why it matters

Our reading: cost per PR measures what a change costs to produce, not what it costs to read. A manager who sees that number fall may miss the people at the other end. Reviewers now face more code, in bigger pieces, and they are the check before it ships.

Anyone buying AI coding tools can act on this. Track review effort next to cost per PR. Ask who is reading the extra code and how long it takes. Jamison's talk, as excerpted, gives no figure for that.

The other side

Jamison was clear about limits. The show notes describe his comparisons as observational. They do not prove that token spending caused the throughput gain, or that companies saw matching cash savings.

The review burden itself is untested here. He mentioned the stress in passing and gave no measurement. His quality check was also reassuring: he saw no sign of serious quality problems with higher spending and faster throughput. Revert rates, meaning PRs that undo earlier ones, moved only slightly, too small to tell apart from random variation.

So the open question is whether review strain is real, how large it is, and whether it ever shows up as a cost. This talk does not answer it.

Written by the WebPulse Newsroom with AI assistance, and checked by our editorial review: every quotation was verified against the recording's transcript. How we use AI.

The conversation this talking point comes from

Share this insight