- Anthropic launched OSS Scanner, a free, opt-in service that sends AI-generated vulnerability reports to open-source maintainers without human review.
- Anthropic says it found over 29,000 candidate flaws but could manually triage only about 6,000. Validation, not discovery, is its limit.
- Leaders should ask which open-source components they depend on, and whether those projects can handle the unvalidated reports they choose to receive.
Checking now trails finding
For years, the scarce skill in security was spotting a flaw. Anthropic's new OSS Scanner suggests a different limit. Anthropic says it can now find candidate flaws faster than people can confirm them.
The company says it has spent six months scanning some of the world's most important software projects with its latest models. That work produced over 29,000 candidate vulnerabilities. These are suspected flaws, not confirmed ones. Anthropic's staff have manually reviewed about 6,000. The company calls human validation its bottleneck.
Anthropic also cites one academic benchmark, CyberGym. On it, models went from finding under 20% of vulnerabilities early last year to over 85% this year. That is a single test, but it fits the picture.
The lesson here is that discovery now outpaces validation. Candidate findings are plentiful. Confirmed and understood findings are not. That is our interpretation of the announcement, not a claim Anthropic makes.
What OSS Scanner does
OSS Scanner is free and opt-in. Core maintainers apply by submitting a pull request to a GitHub repository. Anthropic borrows its test from Google's OSS-Fuzz, which hunts for flaws in open-source code using fuzzers. Fuzzers feed software odd or random inputs to make it fail.
Anthropic looks for projects that matter to infrastructure and user security. It decides case by case.
Help Net Security reports that a new project gets a first round of findings by email. Later scans look for newly added flaws and ones the first pass missed. Maintainers can steer the tool. They can say what to test, which inputs to distrust and how seriously to grade bugs.
No person reviews the output. Anthropic says that makes scanning faster and more frequent. It also means some reports may be wrong.
How a report is built
Each report includes a reproducer. That is a small test that triggers the bug, so an engineer can confirm it fast. The report explains the flaw and, where possible, traces when it was introduced. A candidate patch is added when one exists.
In early trials, Anthropic says some bugs could be chained into unauthenticated remote code execution. That means an attacker with no login could run commands on the software. Anthropic also says exploits can now be built in minutes. Projects that fix flaws faster gain ground on attackers hunting the same weaknesses.
How reliable are the reports?
Anthropic tested an early version. Its penetration testers checked 97 critical and high-severity findings across 48 projects. Of these, 85 met the bar for its disclosure process. Eleven were real but repeated other findings. One was a false positive.
These are Anthropic's own figures. They come from an early version and a high-severity sample. Some maintainers told Anthropic that severity ratings can be inflated. Others said the scanner misread the project's threat model.
Maintainers quoted by Anthropic are positive. Todd Ouska of wolfSSL said all but two of 74 reports were valid, and five became CVEs. Anton Arapov of OpenSSL Corporation said the reports were as good as, and sometimes better than, those from people. These quotes appear on Anthropic's own announcement. Ouska said the reports "slotted right into our existing process," so wolfSSL already had a process to fit them into.
Who carries the work
Some maintainers have asked for the unverified reports. Anthropic says that "with increasing frequency," maintainers who receive its first reports ask for everything it has, unverified ones included. It has sent nearly 5,000 such reports on request. Ouska's verdict on the reports was "We'd love more."
The service stays voluntary. Anthropic points it at teams that already handle confirmed serious reports and have room for more. It will keep sending human-verified reports through its normal process, especially for projects without staff to triage them. Projects can pause the automated reports or opt out.
Help Net Security adds a detail on timing. Unverified findings start no 90-day disclosure clock. The clock may start only if Anthropic later confirms a finding and tells the maintainers.
The main risk sits with projects that opt in without enough people to sort the reports. The announcement does not say how many that is. Even well-staffed projects must check reports that may be wrong or inflated. Help Net Security notes that maintainers remain responsible for checking findings and prioritizing fixes.
Fixes to shared code also reach everyone who runs it. That includes companies that fund none of the work.
What to ask your team
First, ask which open-source components your products depend on, and who maintains them. Second, ask whether your team can apply a patch quickly once a maintainer ships one. Faster upstream fixes help only if you install them.
Third, ask whether you support the projects you rely on, with money or engineering time. If AI gives maintainers more reports to check, their review capacity becomes a risk you share.
Candidate flaws are now plentiful. Confirming and fixing them still takes people.
Produced by the WebPulse Newsroom with AI assistance from the original reporting credited below, and checked against that source by our editorial review. How we use AI.
Original reporting: Anthropic.
