Reproduce “Reducing the Costs of Proof Synthesis on Rust Systems by Scaling Up a Seed Training Set” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Nongyu Di, Tianyu Chen, Shan Lu, Shuai Lu, Yeyun Gong, Peng Cheng, Jacob R. Lorch, Yuan Yao and 1 more
VeruSyn, a data synthesis pipeline, generated 6.9 million Verus-verified Rust programs, establishing the largest dataset for formal verification. This extensive dataset enabled fine-tuning a Qwen2.5-Coder-32B-Instruct model to achieve proof synthesis accuracy comparable to Claude Sonnet 4.5 on system proofs, while reducing verification costs by over 50 times.
Key insights
- A combined data synthesis approach, integrating high-volume, broad-coverage data with complex agent-generated chain-of-thought trajectories, effectively scales LLM capabilities for formal verification.
- Smaller, more cost-effective LLMs can achieve formal proof synthesis capabilities comparable to, or exceeding, larger commercial models when trained on a sufficiently large and diverse synthetic dataset.
- The quality of the 'teacher model' used for generating complex reasoning trajectories significantly influences the performance of the fine-tuned proof synthesis model.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.