Reproduce “Agent-G$^2$: Gaussian Guidance for Agentic Reinforcement Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zixuan Wang, Yanrui Miao, Zhengxi Lu, Teng Pan, Yiwen Qiu, Hongxing Li, Peng Qiu, Ruiqing Zhang and 1 more
Agent-G$^2$ introduces a novel Gaussian guidance framework for training Large Language Model agents in long-horizon, sparse-reward tasks, modeling guidance depth as a per-task Gaussian distribution. This approach adapts guidance parameters online from existing rollouts, outperforming traditional hint-based methods and reducing computational cost compared to probing techniques on benchmarks like ALFWorld and WebShop.
Key insights
- Empirical analysis reveals that useful guidance for hint-based RL exists within a continuous "band" of depths, rather than a single optimal point, and its informativeness profile is approximately Gaussian.
- Dynamically adapting the guidance distribution's center and spread based on policy progress and task-specific difficulty significantly enhances learning effectiveness.
- Computational efficiency can be maintained by estimating per-task guidance parameters online from the same rollouts used for policy optimization, avoiding expensive dedicated probing rollouts.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.