Reproduce “Regularized Score Centering” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hao Bai, Chenlu Ye, Xuanchang Zhang, Aviral Kumar, Tong Zhang
The authors introduce Regularized Score Centering (RSC) to reduce policy mismatch in asynchronous agent training. RSC retains score centering’s reward-baseline correction and penalizes the learner for shifting next-token choices from the data-generating policy, optionally with truncated importance sampling. Across six tasks, RSC with TIS has the highest mean on five and ties on WebShop; results do not establish faster training.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.