Reproduce “The Illusion of Diversity: Aligning LLM Exploration via Effective Entropy” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Xiaoliang Fu, Jiaye Lin, Yangyi Fang, Cong Qin, Binbin Zheng, Chaowen Hu, Zekai Shao

The authors introduce VEPO, a method for reinforcement learning with verifiable rewards that adds effective entropy to Group Relative Policy Optimization (GRPO). Effective entropy measures uncertainty among verifier-accepted outputs, directing an auxiliary exploration incentive toward under-explored valid answers. On 7B DeepSeek-R1-Distill-Qwen, VEPO raises six-benchmark averages to 61.1 Avg@32 and 78.5 Pass@32; matched-count analysis finds 0.37 more strategy clusters per eligible MATH500 problem among answer-verified outputs.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.