Reproduce “greCAPTCHA: Assessing Understanding as Evidence of Research Authorship Under Generative AI” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Justin Payan, Bálint Gyevnár, Atoosa Kasirzadeh, Nihar B. Shah
greCAPTCHA assesses researchers’ capacity to explain and evaluate claims, methods, and evidence in manuscripts tied to their stated contributions. It generates contribution-specific questions and rubrics for a proctored exam. In a study of 31 researchers, scores distinguished tests on participants’ own papers from unfamiliar papers, with ROC AUC 0.90; this supports evidence of understanding, not proof of authorship or correctness.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.