Reproduce “Expert-Guided Rubrics as Dense Reward Signals for Reinforcement Learning in Non-Verifiable Domains” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Ashish Bansal, Hossam Amer, Keerat Kaur Guliani, Parinita Edke, Lesley Matheson, Leizer Teran

The authors train a query rewriter with GRPO using a reward that blends retrieval ranking and an LLM judge’s rubric, distilled from 100 expert-graded question–rewrite pairs. On FlashRAG’s Natural Questions task, adding the rubric to dense retrieval reward raises R@3 from 71.94 to 74.00 and MRR@3 from 0.6296 to 0.6551; results apply to this task.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.