Reproduce “Tool-MAD: A Multi-Agent Debate Framework for Fact Verification with Diverse Tool Augmentation and Adaptive Retrieval” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Seyeon Jeong, Yeonjun Choi, JongWook Kim, Beakcheol Jang
Tool-MAD lets debating agents revisit evidence by using opponents’ answers to formulate searches. One agent retrieves from a Wikipedia-based vector index, while the other uses live web search; evidence-support and question-relevance scores help govern debate and judging. With GPT-4o-mini, it averages 71.0% exact match across four fact-verification datasets, versus 68.0% for MADKE, a benchmark result, not proof of real-world reliability.
Key insights
- Combining diverse external tools and enabling iterative, dynamic evidence retrieval significantly improves factual grounding and performance over static evidence pools.
- Adaptive query formulation, where agents refine queries based on opponent arguments, leads to more targeted and relevant evidence acquisition, particularly for complex claims.
- Incorporating quantitative stability scores (faithfulness and answer relevance) provides crucial internal feedback, enabling real-time detection of factual inconsistencies and improved debate convergence.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.