Reproduce “AutoScaffold: AI-Guided Scaffolding for Reinforcement Learning of Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zijian Zhang, Rizhen Hu, Jiangweizhi Peng, Shiyang Li, Sujay Bhatt, Sihan Zeng, Mingyi Hong
AutoScaffold uses a frozen teacher to inspect reinforcement learning outcomes and revise temporary prompt guidance, which students do not receive at evaluation. On ALFWorld, its 1.5B student achieves 93.9% scaffold-free success, versus GiGPO’s 86.7%. In math, it reports 62.4% average accuracy after 350 student steps, versus QuestA’s 61.1% after 1,100; this is not a wall-clock comparison, and the theory does not explain scaffold-free transfer.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.