Reproduce “Teaching Code LLMs to Reason with Intermediate Formal Specifications” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Minh Le-Anh, Cuong Chi Le, Tien N. Nguyen
SpecCoder trains code models to generate executable assertions at intermediate program states, using execution feedback from correct runs and behavior-changing mutants. On HumanExec, the 14B model’s completeness rises from 0.1989 to 0.7713, while correctness rises from 0.5227 to 0.6468 and assertion validity reaches 1.0000. These results measure performance on available tests and mutants, not proof of correctness or coverage of all bugs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.