Reproduce “Understanding When Reflective Prompt and Skill Optimization Helps on Locally Served Reasoning Models: A Two-Model Study of GEPA and SkillOpt” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Andrea Giummarra, Carmelo Baglieri, Antonio Di Maria, Andrea Farruggio, Michele Piatto, Giovanni Micale

The authors test whether text-based prompt optimization improves locally served models’ benchmark answers without changing model weights. GEPA rewrites prompts from execution traces, and SkillOpt edits a skill document. On Qwen3.8-27B’s 250-item MATH-500 partition, GEPA raises exact match from 0.828 to 0.904, with equal completion counts; gains do not extend to every task or establish improved reasoning.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.