Reproduce “Learning to Coach for Experiential Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Guanheng Chen, Tianzhu Ye, Li Dong, Xun Wu, Shaohan Huang, Furu Wei
L2C trains a separate coach to distill an actor’s long, potentially erroneous history into concise guidance while keeping the actor fixed. The coach receives reinforcement-learning feedback from a verifier judging the actor’s guided response, using same-instance or cross-instance rewards. On FrozenLake with Qwen3-1.7B, same-instance accuracy reached 65.4%, versus 36.5% for Self-Refinement; this result is task- and protocol-specific.
Key insights
- The best experience is not necessarily a faithful transcript: effective coaching filters, corrects, and compresses prior attempts into guidance that the actor can use.
- Cross-instance rewards produce more transferable knowledge in environments with shared mechanics, such as FrozenLake and Sokoban, whereas same-instance training tends to learn map- or problem-specific advice.
- Repeated coaching can outperform simply extending one generation, improving accuracy while shortening responses and adapting the guidance to the actor’s reasoning behavior.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.