Reproduce “What and Whose Knowledge? Measuring Epistemic Diversity in Large Language Models” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Dustin Wright, Sarah Masud, Jared Moore, Srishti Yadav, Maria Antoniak, Peter Ebert Christensen, Chan Young Park, Isabelle Augenstein

The authors measure how varied factual claims are across 27 language models and 155 topics. They group claims with the same meaning and use Hill-Shannon diversity (HSD), which reflects the effective number of equally likely meaning-classes. Scores generally rise across releases, and retrieval increases diversity in the tested setup. GPT-5 averages 2,621 HSD versus search’s 3,110, a lower-bound comparison for these topics and this search baseline.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.