Reproduce “Robust Multi-Agent LLMs under Byzantine Faults” on OpenResearch

About this paper

Haejoon Lee, Vincent-Daniel Yun, Dimitra Panagou, Sai Praneeth Karimireddy

Self-Anchored Consensus (SAC) adapts Byzantine fault-tolerant distributed consensus to multi-agent LLM systems by having each receiver evaluate peer responses against its own answer, rather than trusting sender-reported confidence. Across mathematical, commonsense, and reasoning benchmarks, SAC generally preserved or improved reliable-agent performance, while CP-WBFT often suffered large accuracy losses when an adversary falsified its confidence; for example, on AIME 2025 SAC retained 58.6–61.4% final accuracy versus 0–21% for CP-WBFT.

Key insights

• Moving confidence assessment from the sender to the receiver removes a direct attack mechanism exploited by adversarial agents that claim unjustifiably high confidence.
• Graph robustness is treated as a design requirement for decentralized LLM collaboration: filtering should remove malicious influence without isolating reliable agents or fragmenting information flow.
• SAC’s guarantees concern message retention and propagation rather than answer correctness; performance still depends on the evaluator’s calibration and ability to recognize reliable content.

Summary by alphaXiv.