- SRHarness recovered 93.69% of formulas on the LSR-Transform benchmark with DeepSeek-v4-flash, per the authors.
- On MechBench's core set, Codex with GPT-5.6-sol recovered laws 35% of the time but mechanisms 13.75%. All figures come from the authors' own papers.
- Ask vendors how often their agent is right about the cause, and whether its working evidence is stored where a reviewer can inspect it.
A formula that fits is not an explanation
An AI system can hand you an equation that matches every number you have. That does not mean it knows why the numbers behave that way. Researchers at Tsinghua University have now built a test that measures this gap.
The team is the Data Science and Intelligence Lab in the Department of Electronic Engineering. It released two linked projects, as reported by Pandaily on October 14. SRHarness is open-source software that helps AI agents search for formulas. MechBench is a benchmark that asks whether an agent can recover the mechanism behind a formula, not only the formula.
The researchers frame it with history. Kepler described how planets move from observations. Newton explained why. They wanted to test an AI on both steps.
How SRHarness works
The task is called symbolic regression. The agent searches for a readable mathematical formula that fits a set of data.
SRHarness does not dictate a search strategy. It supplies the plumbing instead, in three parts. First, one shared interface lets the agent call tools for analysis, curve fitting, search and scoring. Second, candidate formulas and the evidence behind them sit in a store outside the chat. The likely point of that design is that a long conversation can drift or run out of room. Third, the system manages the life of a long search, deciding whether to continue, branch, restart or stop.
The code is open source on GitHub and installable from PyPI. There is also an online demo.
The authors report that the earlier method SR-Scientist reached 62.16% with the same base model. They say that with scientific context and variable meanings removed, SRHarness still recovers 72.97% of target equations. SR-Scientist reaches 39.64% and Codex on the same base model reaches 20.72%. The authors add that SRHarness also beats a setup pairing Codex with GPT-5.5, which scored 68.47%.
What MechBench adds
Many different mechanisms can produce identical observations. A formula alone cannot tell them apart. MechBench uses mechanism probes to check internal quantities that a formula cannot determine. It also screens competing mechanisms that would give the same results.
The result is sobering. In the paper, Codex running GPT-5.6-sol identified the underlying mechanism in just 13.75% of MechBench core-set cases.
Read these two figures together. They come from the same benchmark and the same agent. The 93.69% above comes from a different benchmark and a different setup, so it should not be set against 13.75%.
The researchers draw a wider conclusion from the gap. Their claim is that agents built for science today are strong at spotting regularities and weak at grasping what produces them. They present MechBench as a way to measure progress on that second skill.
Why this matters beyond the lab
What follows is interpretation, not the authors' claim. Leaders are being sold agents that analyze data and recommend actions. A recommendation built on a pattern is a forecast. It holds while conditions resemble the data. A recommendation built on a mechanism can say what happens when conditions change.
MechBench shows that the two can be separated and measured. A strong score on finding answers says little about explaining them. Demos tend to show the first, so ask for the second.
The design of SRHarness also carries a lesson for governance. Because candidate formulas and their supporting evidence live outside the conversation, a reviewer can go back and check what the agent tried. That is a useful property for any agent whose work you may need to audit.
Limits of what we know
These figures come from the authors' own papers, reported by Pandaily. We have not seen independent replication. The tests cover formula discovery on specific benchmarks. They do not show how agents perform on business data.
Questions to put to your team
Ask any vendor selling analytical agents these four things. Does the system report how often it is right about the cause, not only the answer? Was it tested on cases where several explanations fit the same data? Is its working evidence kept outside the chat, where a reviewer can read it? Who signs off when the agent finds a pattern and no one has checked the reason?
A formula tells you what to expect. A mechanism tells you when to stop expecting it.
Produced by the WebPulse Newsroom with AI assistance from the original reporting credited below, and checked against that source by our editorial review. How we use AI.
Original reporting: Pandaily.
