Reproduce “Self-signals Driven Multi-LLM Debate for Efficient and Accurate Reasoning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xuhang Chen, Zhifan Song, Deyi Ji, Shuo Gao, Lanyun Zhu
The paper introduces Self-Signals Driven Multi-LLM Debate (SID), a framework that leverages intrinsic model confidence and token-level semantic focus to improve multi-agent reasoning. SID achieves up to a 40% reduction in token usage and higher accuracy across various LLM and MLLM benchmarks by dynamically managing debate participation and context.
Key insights
- Intrinsic self-signals from LLMs, such as token logits and attention weights, can effectively guide and optimize multi-agent debate processes more robustly than external mechanisms.
- Model-level confidence, quantified through uncertainty metrics, enables dynamic early-exiting, significantly reducing computational overhead without compromising accuracy.
- Prompt-conditioned attention allows for precise extraction of semantically focused spans from debate history, leading to adaptive compression that maintains crucial information while minimizing redundancy.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.