Reproduce “Self-Trained Verification for Training- and Test-Time Self-Improvement” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chen Henry Wu, Aditi Raghunathan
Self-trained verification (STV) trains a model to judge attempted solutions and give feedback, using a teacher that sees reference solutions during training; the verifier does not see them at inference. With Qwen3-8B, STV raises pass@1 from 1.5% to 21.0% on Hardest SciKnowEval. ViL training also raises round-0 math pass@1 from 10.7% to 14.7%, in experiments limited to hard benchmarks.
Key insights
- An asymmetry exists where LLMs can more reliably diagnose errors in a candidate solution when presented with a ground-truth reference, allowing for the generation of high-quality diagnostic self-supervision.
- Providing specific, diagnostic feedback is more effective for iterative refinement and generator training than merely providing a binary correctness signal or generic critiques.
- Training a generator to actively learn from diagnostic feedback instills more generalized reasoning capabilities, leading to improvements in its standalone performance beyond what traditional RL methods achieve.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.