Reproduce “Exploring Communication Strategies for Collaborative LLM Agents in Mathematical Problem-Solving” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Liang Zhang, Xiaoming Zhai, Jionghao Lin, Jennifer Kleiman, Diego Zapata-Rivera, Carol Forsyth, Yang Jiang, Xiangen Hu and 1 more

This study examines how different communication strategies between Large Language Model (LLM) agents affect their ability to solve challenging mathematical problems. It finds that structured dual-agent communication outperforms single-agent approaches, with peer-to-peer collaboration achieving the highest accuracy of 54.10% on level-5 MATH dataset problems. The research also details the specific dialogue acts that contribute to these performance improvements.

Key insights

  • Collaborative dual-agent LLM systems consistently achieve higher problem-solving accuracy compared to single-agent LLMs on complex mathematical tasks.
  • The specific structure and dynamics of communication, as reflected in the patterns of dialogue acts, are critical factors influencing the effectiveness of collaborative LLM agents.
  • Core dialogue acts such as statements, hints, and acknowledgments play essential roles in facilitating reasoning and maintaining effective collaborative dynamics across different communication strategies.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.