Reproduce “SWE-Router: Routing in Multi-turn Agentic Software Engineering Tasks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Seongho Son, Sangwoong Yoon, Jiahua Tang, Shuhan Wang, Lorenz Wolf, Ilija Bogunovic
SWE-Router lets a cheaper model explore a repository before choosing whether the cheaper model continues or a stronger one takes over. A value head uses the trajectory to estimate the cheaper model’s chance of resolving the issue. On 100 held-out SWE-Bench Verified issues, K=2 Route-AUC was 0.780 versus 0.627 for K=0, a 15.3-point gain, supporting benefits on this slice, not broad generalization.
Key insights
- Dynamic information from partial execution trajectories significantly improves the accuracy of routing decisions for multi-turn agentic tasks, surpassing the limitations of prompt-only approaches.
- Incorporating more informative signals, such as intermediate observations, can theoretically enhance Bayes-optimal routing for cost-benefit utility.
- Strategic routing can unlock a synergistic effect where combining weaker and stronger models through informed switching achieves a higher overall resolution rate than either model used in isolation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.