Reproduce “SPEAR: Code-Augmented Agentic Prompt Optimization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mengyin Lu, Cong Feng, Huimin Han, Guangming Lu, Yu Sun, Xiaonan Ding, Shihui Long, Fengyi Li and 1 more
SPEAR lets an agent optimize prompts by writing Python to analyze predictions, group errors, and choose revisions, rather than relying on feedback loops. On Banking77 classification, it reached Cohen’s κ of 0.927±0.014, versus GEPA’s 0.816±0.009. On Hiring Assistant location judging, its seven-run mean was 0.483 versus 0.260; evidence is strongest for structured judge errors, not prompt optimization generally.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.