Anthropic Uses Claude to Automate Mathematical Safety Checks for Software

Anthropic has demonstrated a significant step in software reliability by using its Claude model to automatically generate mathematical proofs of code correctness. Working alongside partner Theorem, the company showed that AI can now handle the rigorous task of formal verification, a process that has historically been too expensive and slow for widespread use. This development suggests that ensuring software safety could soon become a routine part of the development cycle rather than a specialized luxury reserved for critical systems.

The need for this technology stems from the high cost of traditional verification. In the past, proving that software runs without errors under all conditions required extensive human effort. For example, a team of specialists needed twenty person-years to formally verify the seL4 operating system kernel. Because of this heavy resource requirement, formal verification was typically limited to extreme cases, such as the avionics software used by Airbus. The goal of the new approach is to make this level of assurance accessible for a broader range of software applications.

The recent project involved Claude working with test programs from 123 open-source libraries included in Google's OSS-Fuzz security project. Instead of writing new software from scratch, the model focused on verifying existing code. It achieved this by translating crash tests into mathematical problems and analyzing the compiled machine code. Over the course of just a few weeks, Claude produced approximately 180 million lines of proof code. An independent program called Rocq automatically checked these proofs, and the process required almost no human intervention, leading researchers to describe it as fully autonomous.

One of the most notable outcomes of this work was the discovery of numerous hidden defects. While generating proofs, Claude identified hundreds of bugs and was permitted to fix them. Many of these issues would have escaped conventional testing because they only appear under very specific conditions, such as unusually large input files or particular sequences of data access. In one specific instance, the team found as many security bugs in a widely used PDF reader in a single week as are typically reported for that software over an entire year. In another case, the model caught a logical error that had remained undetected for more than a decade.

Despite these successes, there are important limitations to what this method can achieve. Anthropic notes that being formally verified does not mean the software is completely error-free. The proofs hold only under strict conditions, including a specific processor architecture, single-threaded programs, and inputs smaller than 64 megabytes. In real-world scenarios, these libraries often run on different hardware and configurations. Additionally, the computational cost is significant, with individual proofs consuming hundreds of millions of tokens, though the company has not disclosed the total project costs.

The efficiency of the process is improving as newer models are released. A smaller test run indicated that proofs are reusable, reducing the need for repeated calculations. Newer iterations of the model, such as Opus 5.5, are also more efficient. This version solved half the proofs using about 40 percent fewer tokens than the previous Opus 5 model. These improvements suggest that the cost of verification will continue to drop as the technology matures.

The implications of this project extend beyond simple coding assistance. If AI can write code and also guarantee its correctness under defined conditions, the industry can place greater trust in AI-generated software. This could lead to significant savings in both verification and the broader coding process. The shift moves the focus from whether AI can program to whether it can prove that its code is safe. If the answer remains yes, the impact could reach well beyond current development tools.

Looking ahead, Anthropic expects formal verification with AI to become as practical as standard testing in the near future. Theorem's roadmap already includes plans to apply this method to critical infrastructure components such as glibc, OpenSSL, and cURL. This expansion indicates a growing confidence in using AI to secure the foundational layers of software that power modern computing.