Reproduce “Density-Aware Reward Scaling: A Reinforcement Learning Framework for Reward-Proportional Sampling” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Ria Shekhawat, Abhijeet Sinha, Yingtao Zhu, Dianbo Liu

DARS addresses repetitive outcomes in reward-optimized generation by dividing task rewards by local density within each on-policy rollout group, then applying the usual GRPO update. For images, it estimates density from frozen DINOv2 features. In navigation, mode-TV error was 0.014 versus 0.750 for GRPO; image diversity also increased in one condition, but image metrics are proxies rather than direct evidence of recovering the target distribution.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.