Reproduce “The Illusion of Diversity: Aligning LLM Exploration via Effective Entropy” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xiaoliang Fu, Jiaye Lin, Yangyi Fang, Cong Qin, Binbin Zheng, Chaowen Hu, Zekai Shao
The authors introduce VEPO, a method for reinforcement learning with verifiable rewards that adds effective entropy to Group Relative Policy Optimization (GRPO). Effective entropy measures uncertainty among verifier-accepted outputs, directing an auxiliary exploration incentive toward under-explored valid answers. On 7B DeepSeek-R1-Distill-Qwen, VEPO raises six-benchmark averages to 61.1 Avg@32 and 78.5 Pass@32; matched-count analysis finds 0.37 more strategy clusters per eligible MATH500 problem among answer-verified outputs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.