Reproduce “Supplement Generation Training for Enhancing Agentic Task Performance” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Young Min Cho, Daniele Bonadiman, Divya Bhargavi, Tamer Alkhouli, Salvatore Romeo, Dongwei Jiang, Khushbu Pahwa, Yubin Ge and 3 more
Supplement Generation Training (SGT) introduces a framework where a small language model is trained to dynamically generate instance-specific textual supplements, guiding a larger, frozen foundation model to enhance its agentic task performance. This method achieved an average performance gain of 21% over baseline models across diverse benchmarks, particularly improving structured reasoning tasks, while demonstrating computational efficiency.
Key insights
- Training a small language model specifically to generate context-rich supplements for a larger, frozen LLM dramatically improves task performance, outperforming direct fine-tuning of the small model as a solver.
- Iterative Direct Preference Optimization (DPO) significantly enhances supplement quality and effectiveness beyond initial supervised fine-tuning (SFT), leading to greater gains.
- The optimal type of generated supplement is highly task-dependent, with structured reasoning tasks benefiting most from comparative examples like 'Pairs' and code tasks from 'Summary' and 'Step-by-step reasoning'.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.