Reproduce “When AI Reviews Train AI Reviewers: Scientific-Judgment Collapse and Mitigation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Sy-Tuyen Ho, Minghui Liu, Hieu Tran, Furong Huang

The authors test whether AI-generated reviews shape later AI reviewers by varying explicitly model-generated reviews in successor training while holding planned conditions fixed. Greater exposure concentrated ratings and reduced embedding-based diversity; same-paper semantic distance fell from 0.159 to 0.142. These measures do not establish reduced review quality, and official reviews may include unobserved AI assistance.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.