Reproduce “Aligning Training and Rollout Policies for End-to-End FP4 RL on Mixture-of-Experts LLMs” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Hei Yi Mak, Hoang Le, Mehran Taghian Jazi, Yunke Peng, Huan Lin, Zhengzhi Tang, Zhexu Liu, Yao Wang and 3 more

The authors evaluate MoEResQ, which applies residual correction symmetrically during rollout and training for FP4 reinforcement learning with mixture-of-experts models. They compare it with rollout-only correction and measure policy alignment and math-benchmark accuracy. On AMC23, expert-only MoEResQ reaches 84.61 accuracy versus 78.75 for rollout-only correction and 85.10 for BF16; quantization is simulated, so efficiency gains are not measured.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.