Reproduce “ReGround: Grounding Reviewer Comments in Multimodal Evidence” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Serwar Basch, Lizhen Qu, Iryna Gurevych

ReGround introduces a benchmark for linking implicit reviewer comments to the paragraphs, sections, figures, and tables that support authors’ rebuttal responses. Built from 3,656 original submissions and 10,267 comments, the dataset shows that unified evidence retrieval remains difficult, while type-aware and multimodal retrieval can substantially improve performance.

Key insights

  • Mixed evidence-type retrieval is the primary bottleneck: the best unified system achieves only 5.12% Recall@1 and 21.15% Recall@10, whereas oracle knowledge of the evidence type yields much higher recall.
  • Automatically hard-filtering candidates using predicted evidence types can hurt performance by eliminating the correct unit; evidence-type predictions are better used as soft ranking signals.
  • Figures and tables provide information beyond their captions, and image-based and text-based retrieval often identify different relevant examples, supporting multimodal fusion.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.