- METR, a group that tests frontier AI, published a proposal on October 14, 2026 for checking whether AI is speeding up AI research.
- The check needs internal data and researcher interviews that outsiders lack. METR warns that benchmarks and self-reported speedups can both mislead.
- Leaders can use the same method on their own AI claims: dated baselines, separated inputs and a stated alternative.
Who checks the claim when a technology helps build its own next version? The people who see the change first are the ones making it. METR, a group that tests frontier AI, took up this problem in a proposal published on October 14, 2026. The lesson for executives is about evidence. A claim about AI speed is only as good as the data behind it.
A plan for checking, not a finding
METR researches and tests frontier AI systems. Its post describes how an outside assessor could study one question. Could AI progress speed up sharply because AI is doing more of the work of AI research?
This is a proposal. It reports no result. It does not say that such a speedup is happening.
The check rests on four questions. How fast are internal capabilities improving, and has that sped up? If so, how much came from AI automation and how much from other causes? Could any bottleneck slow growth soon? And if progress had jumped far more sharply, would these methods have caught it?
The hard part is finding the cause
Faster progress alone proves little, because other inputs can cause it. METR gives one example. A bigger-than-trend jump in compute (computing power) could lift capabilities with no help from AI automation.
So the assessor would track four inputs: labor, compute, data and algorithms. Compute splits in two. One part is the training compute used for each model. The other is experimental compute over time, by team and function.
Algorithms are harder to put a number on. METR expects developers already keep estimates that map each research advance to a reduction in the compute needed for a given level of performance. Developers have not published their methods. The assessor would ask for those estimates, dated precisely, not the advances themselves.
The hardest step is the counterfactual. That means working out what would have happened without AI. METR says this is challenging, so a good estimate needs several lines of evidence.
METR lists examples. One is spend on AI agents, counted in tokens and inference cost. Others are labor productivity measures, time-use logs from researchers and accounts of fully autonomous discoveries.
That survey covers technical workers broadly. It is not about AI research speedups. The median self-reported change in the value of work from AI tools was 1.4 to 2x. METR says there are reasons to doubt the size of that gain. Read it as a sample of self-reports, not proof that AI research is faster.
Every single measure can mislead
METR names the weak spots itself. Benchmark scores may overstate what a model can do, because passing a test is not the same as being useful in real work. Researchers' own speedup estimates may overstate it too, because the no-AI case is hard to judge.
So no single signal is enough. The proposal combines benchmarks, early indicators such as pretraining loss, algorithm estimates, input data and interviews. It also asks how the internal evidence compares with public evidence, and what explains any gap.
Access is the real limit
Much of the needed data is not public. It includes scores for internal-only models, dated progress records, team sizes, compute use and agent spend. Assessors would also interview staff in pretraining, reinforcement learning and infrastructure.
METR suggests the data could be analyzed on the developer's own hardware, possibly on site. Only minimal access to the developer's systems is needed. METR also warns that forward-looking estimates are highly uncertain. Reports should say so.
What leaders should ask
This is WebPulse's view: the proposal is a useful template for judging any AI claim. It applies whether the claim comes from a model developer, a supplier or your own teams.
Put four questions to suppliers and internal teams. First, what dated baseline exists from before the AI tool was adopted? Second, what else changed at the same time, such as headcount, budget or tools? Third, is the productivity figure measured or self-reported? Fourth, how was the no-AI case handled?
A claim that cannot answer those questions is closer to opinion than evidence. METR's proposal shows what the stronger kind looks like: separated inputs, precise dates, several independent measures and stated limits.
Produced by the WebPulse Newsroom with AI assistance from the original reporting credited below, and checked against that source by our editorial review. How we use AI.
Original reporting: METR.
