Reproduce “From Talking Words to Sharing Thoughts: Scalable Multi-LLM Aggregation via Structured Message Passing” on OpenResearch

About this paper

Niloufar Mehrabi, Sayed Pedram Haeri Boroujeni, Abolfazl Razi

Researchers at Clemson University developed 'From Talking Words to Sharing Thoughts,' a logit-level aggregation framework that efficiently combines multiple Large Language Models (LLMs) to enhance predictive accuracy. This method employs structured message passing on a factor graph with adaptive damping, achieving state-of-the-art results on several challenging benchmarks while significantly reducing computational costs and mitigating anchor corruption.

Key insights

• Collaborating at the logit layer, rather than through iterative natural language exchanges, dramatically improves efficiency and scalability by preserving epistemic information.
• Dynamically adapting the aggregation process based on the detected epistemic state and domain-specific expertise enhances the robustness and accuracy of the ensemble.
• Asymmetric damping, guided by model reliability and confidence, is crucial for mitigating anchor corruption and effectively leveraging diverse LLM strengths.

Summary by alphaXiv.