Reproduce “Where Is the Truth When AI Agents Disagree? Dependence-Aware Belief Propagation for Unsupervised Truth Discovery (Extended Version)” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Laure Berti-Equille
The authors introduce BP-TD, which discounts votes from agents sharing a base model or agreeing unusually often. Its heuristic weights enter a Dawid–Skene-style procedure that alternates answer selection and relative reliability estimation. On 10,000 MMLU questions answered by 13 agents, label-free family BP-TD achieved 0.707 accuracy versus 0.674 for majority vote; gains are not guaranteed across pools or tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.