Reproduce “Table-Critic: A Multi-Agent Framework for Collaborative Criticism and Refinement in Table Reasoning” on OpenResearch

About this paper

Peiying Yu, Guoxin Chen, Jingjing Wang

Researchers from Soochow University and the Chinese Academy of Sciences developed Table-Critic, a multi-agent framework that uses collaborative criticism and an evolving knowledge base to enhance large language models' accuracy in complex table reasoning tasks. The framework consistently achieved 73.7% accuracy on WikiTableQuestions and 91.7% on TabFact, marking improvements of 6.3% and 2.2% respectively over strong baselines, while significantly reducing solution degradation.

Key insights

• LLMs are proficient at identifying initial errors but may introduce new ones in subsequent steps, necessitating a multi-turn, sequential refinement process that rebuilds the reasoning from the error point.
• The diverse and unpredictable nature of errors in complex reasoning requires a dynamic knowledge base, where critique patterns are systematically accumulated and categorized to improve error identification over time.
• Distributing the tasks of error identification, critique generation, and refinement among specialized agents, rather than relying on a single LLM's self-reflection, leads to more robust and reliable error correction.

Summary by alphaXiv.