Reproduce “LoGRA: Scaling LLM Reinforcement Learning with Low-Rank Gradient Sketches” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shaokun Zhang, Yifan Zhang, Jian Hu, Yueying Li, Hao Zhang, Binfeng Xu, Jan Kautz, Yi Dong
LoGRA reduces reinforcement-learning training memory by directly accumulating compact sketches of selected weight gradients, then using them to update full-sized weights and synchronize the rollout policy. On Qwen2.5-Math-7B, average memory per GPU fell from 31.82 GiB with dense Adam to 17.29 GiB, while peak Pass@1 was similar: 72.48% versus 72.33%. This compares reported recipes, not gradient compression alone.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.