Reproduce “Arbiters of Ambivalence: Challenges of Using LLMs in No-Consensus Tasks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Bhaktipriya Radharapu, Manon Revel, Megan Ung, Sebastian Ruder, Adina Williams
Researchers at Meta investigated Large Language Models' behavior in tasks lacking human consensus, demonstrating that while LLMs can generate neutral responses when explicitly prompted, their neutrality significantly diminishes when acting as judges or debaters. The study found that LLMs tend to force a definitive stance in these no-consensus scenarios, deviating from inherent human disagreement patterns and exhibiting higher divergence from human judgment distributions when less neutral.
Key insights
- LLMs' neutrality consistently decreases when transitioning from answer generation to judging or debating roles, even on topics where humans inherently disagree.
- The final decision of an LLM judge in a debate is remarkably stable and consistent, regardless of the specific LLMs acting as debaters, suggesting inherent preferences or biases within the judge model itself.
- Explicitly providing a "neutral" or "both" option in the prompt significantly increases the likelihood of LLMs adopting a neutral stance as generators.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.