Reproduce “Fail-Fast, Restart-Smart: Early Failure Prediction and Restart for SWE Agentic Tasks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chenyu Wang, Yunbo Lyu, Junda He, Zhou Yang, Chenxing Zhong, Yaniv Harel, David Lo
An integrated system, FailFast–RestartSmart, predicts and terminates failing Large Language Model (LLM)-based software engineering (SWE) agent trajectories early to save compute, then initiates a fresh attempt using prior edits as an unverified overlay to boost overall task resolution.
Key insights
- Effective failure prediction for agentic tasks can be achieved by monitoring observable trajectory prefixes, making the approach broadly applicable to both open-weight and closed-API LLMs without requiring internal policy signals.
- Providing salvaged edits as an *optional, unverified overlay* allows agents to leverage partial progress from failed attempts while maintaining critical evaluation, preventing the negative anchoring effects of direct prompt injection.
- Training the failure monitor with dense Fail-to-Pass progress targets and Bradley–Terry preference pairs significantly improves its ability to predict failure compared to relying solely on sparse terminal outcomes.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.