Reproduce “How to Train Your LLM Web Agent: A Statistical Diagnosis” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Dheeraj Vattikonda, Santhoshi Ravichandran, Emiliano Penaloza, Hadi Nekoei, Megh Thakkar, Thibault Le Sellier de Chezelles, Nicolas Gontier, Miguel Muñoz-Mármol and 8 more
Researchers from ServiceNow AI Research and Mila-Quebec AI Institute developed a statistically grounded methodology to optimize the training of open-source LLM web agents. Their hybrid SFT+RL approach achieved comparable performance to proprietary models on MiniWoB++ while reducing computational costs by up to 45% compared to pure SFT.
Key insights
- A hybrid SFT+RL training strategy consistently surpasses the performance of pure SFT or pure RL methods for LLM web agents.
- Significant computational efficiency is gained by strategically branching into RL fine-tuning from SFT warm-up checkpoints.
- Optimal hyperparameter settings for RL dynamically change based on the extent of prior SFT warm-up, influenced by factors like entropy reduction.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.