Reproduce “Message capacity and claim wording set the transition points of collective truth-finding in language-model networks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Makoto Fukushima
A mean-field framework predicts when networks of LLM agents reach correct or incorrect consensus from two separately measured factors: message capacity and the individual model’s claim-dependent response function. Experiments show that capacity alone is insufficient when wording induces a strong assertion bias in llama3.1:8b, whereas the same calibration procedure accurately predicts transition points for qwen3:8b.
Key insights
- Message effects weakened as inbox size grew; a divisive-normalization response model reduced fit error from 0.118 for constant weights to 0.054.
- The baseline field associated with a claim’s wording can outweigh message capacity: llama3.1:8b tended to follow the asserted polarity rather than factual truth, causing wrong consensus even when 24 of 32 agents initially held the correct view.
- Model-specific calibration is essential. The framework generalized well to qwen3:8b but the 70B probe did not show the same assertion asymmetry, indicating that collective reliability cannot be inferred from message capacity alone.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.