Reproduce “Phantom Consensus: Measuring Social Reasoning in Multi-Agent LLM Decision-Making” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Disha Sheshanarayana, Rajat Subhra Pal, Manjira Sinha, Tirthankar Dasgupta

The authors introduce PHANTOM-BENCH to test whether ambiguous peer comments lead multi-agent language models to misinfer peers’ private decisions and shift public choices. Four agents first decide privately, then report public decisions and estimates of peers’ private views across 480 controlled scenarios. For Qwen-2.5-7B, accuracy fell from 77.0% to 29.2%; results are benchmark-specific, not deployment-rate estimates.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.