AI now writes security proofs for open-source code, with fine print

Testing finds bugs. A proof draws a boundary.

Security teams have long relied on testing. Throw many inputs at the code, fix what breaks, and accept the risk that remains. Anthropic and the security firm Theorem now show a different kind of assurance for open-source code. It is a machine-checked proof that certain failures do not occur on any input within stated limits.

This points to a change in what is scarce. Proofs were expensive, so almost nobody asked for them. If AI makes them cheaper, the scarce skill becomes reading what a proof actually covers. The fine print becomes the product.

What Anthropic and Theorem did

Anthropic says Claude proved security properties at the assembly level, the machine instructions a program runs. It puts the count at 123 open-source libraries. To its knowledge, it says, this is the first large-scale use of AI to formally verify production software at that level.

Theorem's public repository holds a subset. It contains proofs for 60 entry points in 25 libraries.

Anthropic also says Claude found bugs while writing the proofs. By its account, the bugs had gone unnoticed for years, even after trillions of test runs. The material we reviewed does not name those bugs or libraries.

Anthropic explains why this was out of reach before. It says the seL4 operating system kernel took two person-years to build and 20 to verify. That gap kept formal verification for work such as Airbus's avionics code.

How the proof works

Fuzzing means feeding a program odd inputs to see what breaks. OSS-Fuzz does this for open-source projects, using small test programs called fuzz targets. A sanitizer is an add-on to the build that flags memory errors, such as AddressSanitizer.

Each proof makes an agreement claim. On every input within the limits, the sanitizer build behaves like the plain build, and the sanitizer never detects an error.

The proofs are written in Rocq, a proof assistant, and checked with version 9.2.0. The rules they rely on come from LPS, a formal model of how a Linux process behaves when it runs a RISC-V program. Theorem's checking tool matches each binary to its SHA-256 fingerprint. It also refuses any assumption outside an allowed list. Theorem says its automated checks rerun weekly.

This matters for trust. You do not have to trust the AI that wrote the proof. You trust the checker and the model.

The fine print

Theorem's repository is direct about limits. Every proof is tied to one build recipe: statically linked riscv64 Linux programs, compiled with clang 20.1.8 and the musl C library. A proof about those exact files does not carry over to the source code. It also does not carry over to the x86-64 builds that OSS-Fuzz itself uses.

LPS is best effort. Theorem says it describes only part of Linux and makes some assumptions it does not check. Where the model differs from real Linux, Theorem warns, a proof may not apply.

Each proof covers the code one entry point reaches, not a whole library. It does not say the library is correct. Wrong results and memory leaks are not excluded. No proof in this release rules out a run that never ends.

Anthropic adds its own conditions. Inputs are under 64 MB, and the program is single-threaded with a fixed memory budget. Anthropic says the work is "still far away from fixing all possible bugs."

Early days: 138 proved, 19,797 open

The repository lists statements for 2,148 fuzz targets across 427 buildable projects. OSS-Fuzz has 1,381 projects in total. Of 20,164 statements in this release, Theorem reports 138 proved, 229 blocked by the model and 19,797 open.

Anthropic and Theorem produced this work, and Anthropic's own model wrote the proofs. The sources report no independent audit. Still, the material is open. Each proof has a build record, and Theorem's documentation explains how to rebuild a binary and compare its fingerprint with that record. Others can recheck the proofs.

What to ask your team

First, ask suppliers of important open-source components whether any proof exists. If one does, ask which binary, which entry point and which limits it covers. Theorem cites statements by name and revision, such as zlib/checksum_fuzzer/plain_halts@r1. Revisions are provisional until LPS is released.

Second, check whether the proved build matches what you run. A proof about a riscv64 build does not transfer on its own to an x86-64 build in your production systems.

Third, keep fuzzing, patching and monitoring. A proof about memory errors within limits does not replace them.

Fourth, if your company maintains open-source code, ask whether enrolling in Theorem's OSS Verify program fits your roadmap.

Testing tells you what failed. A proof tells you exactly what was checked. The skill leaders now need is reading what it leaves out.

Produced by the WebPulse Newsroom with AI assistance from the original reporting credited below, and checked against that source by our editorial review. How we use AI.
Original reporting: Theorem (theorem-labs) with Anthropic.