Anthropic and Theorem Automate Code Safety Proofs for Open-Source Libraries

Anthropic has demonstrated that its Claude model can automatically generate formal mathematical proofs to verify the safety of existing software. In a recent collaboration with partner Theorem, the AI produced approximately 180 million lines of proof code in a matter of weeks. This development marks a significant shift in how developers might approach software reliability, moving beyond standard testing to rigorous mathematical validation.

The primary objective of this initiative is to address the high cost and labor intensity of formal verification. Traditionally, proving that software runs without errors under all conditions has been reserved for critical systems, such as the seL4 operating system kernel, which required 20 person-years of specialist work. By leveraging Claude, Anthropic’s Frontier Red Team and Theorem aimed to automate this process, allowing the model to analyze compiled machine code and translate crash tests from 123 open-source libraries in Google’s OSS-Fuzz project into solvable mathematical problems.

The process is described by researchers as "fully autonomous," though it relies on foundational tools built by human experts at Theorem. An independent verification program named Rocq checks the proofs generated by Claude. According to Anthropic, this workflow requires minimal human review. The system does not write new code; instead, it verifies existing software by generating extensive proofs that confirm the code behaves as intended under specific constraints.

During the verification process, Claude identified hundreds of bugs that conventional testing methods had previously missed. Many of these errors only manifested under rare conditions, such as unusually large input files or specific sequences of memory access. For instance, while verifying a widely used PDF reader, the team discovered as many security vulnerabilities in one week as are typically reported for that software over an entire year. In another case, the model caught a logical error that had remained undetected for more than a decade.

Despite these successes, Anthropic notes that "formally verified" does not equate to "error-free" in a universal sense. The proofs are valid only under strict conditions, including a specific processor architecture, single-threaded execution, and inputs smaller than 64 MB. Since these libraries often run on diverse hardware and configurations, the practical scope of these guarantees is limited to defined environments.

The computational cost of this approach remains substantial. Individual proofs can consume hundreds of millions of tokens, and Anthropic has not disclosed the total project expenses. However, the company highlights efficiency gains in its newer models; Opus 5.5, for example, solves half of the proofs with approximately 40% fewer tokens than the previous Opus 5 version. A smaller test run also indicated that the generated proofs are reusable, potentially reducing long-term costs.

If AI can reliably verify its own code, developers may place greater trust in AI-generated software, reducing the time spent on manual reviews. This could lead to cost savings across the entire coding lifecycle. Anthropic expects formal verification using AI to become as routine as standard testing, with Theorem’s roadmap already targeting critical infrastructure components such as glibc, OpenSSL, and cURL. The project suggests that the industry is moving toward a future where AI not only writes code but also provides mathematical guarantees of its safety.

This advancement shifts the central question in software automation from whether AI can program to whether it can prove its code is safe. If the latter becomes the standard, the implications for the technology sector could extend well beyond current coding tools, influencing how critical software is developed and maintained.