Reproduce “Schwarz: Solver-Aware Agentic Program Verification” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jingyu Ke, Ling-I Wu, Guoqiang Li
SCHWARZ is an agentic program verification system that enhances large language model (LLM) agents' ability to repair proof failures by providing granular, solver-aware feedback. The system achieved a 92.7% pass rate across 1,475 diverse C and Rust verification tasks, surpassing existing agentic tools and traditional verifiers.
Key insights
- Solver dischargeability, not just logical truth, is a critical bottleneck in SMT-backed agentic verification that requires explicit mechanisms for LLM agents to address.
- Providing granular, solver-level feedback, including specific failure points, available facts, and suitable theory views, significantly improves LLM agents' ability to diagnose and repair proof failures.
- Embedding expert domain knowledge through "theory-aware policies" guides LLM agents towards efficient SMT formulations, reducing redundant exploration and enhancing proof search effectiveness.
- The design of the trusted harness, exposing actionable solver-level context while maintaining deterministic checking, is crucial for building robust and effective agentic verification systems.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.