Reproduce “Offline Training of Language Model Agents with Functions as Learnable Weights” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Shaokun Zhang, Jieyu Zhang, Jiale Liu, Linxin Song, Chi Wang, Ranjay Krishna, Qingyun Wu

Researchers from Penn State, UW, USC, and Microsoft Research introduce a method for offline training of language model agents by optimizing external functions, treating them as learnable weights. Their approach, integrated into the AutoGen library, utilizes an LLM as an "AgentOptimizer" to iteratively refine function sets, leading to average performance improvements of 6% to 8.5% on tabular processing and general real-world tasks for various agent systems.

Key insights

  • Large Language Models can effectively act as optimizers for symbolic function updates, refining tool sets based on agent performance and execution history.
  • Iterative, progressive refinement of functions through small modifications is crucial for effective agent training, outperforming one-shot function generation.
  • Training stability and efficiency are significantly enhanced by employing strategies such as roll-back (to revert detrimental updates) and early-stop (to conclude optimization when improvements cease).

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.