Reproduce “ConsensusBench: Benchmark of Consensus Nodes for LLM Reasoning via Outcome Reward Densifying” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Shi-Qi Yan, Chao-Hong Tan, Qian Chen, Wen Wang, Xiangang Li, Zhen-Hua Ling

ConsensusBench introduces a benchmark and reward scheme for densifying reinforcement-learning feedback in mathematical reasoning. By identifying intermediate conclusions shared across multiple correct solutions, ConsensusPR improves difficult-problem accuracy for Qwen3-4B and Qwen3-8B over outcome-only GRPO/DAPO training, while also exposing the trade-off between greater process coverage and longer responses.

Key insights

  • Recurring intermediate conclusions provide a form of process supervision that is less tied to a single solution path than canonical step-by-step annotations.
  • Reward densification is most useful on difficult problems with long reasoning trajectories, where terminal rewards create the largest credit-assignment gap.
  • Higher node coverage does not necessarily imply efficient reasoning: process-aware rewards often increase response length and tokens per matched node, so final accuracy and reasoning efficiency must be evaluated together.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.