Reproduce “Self-DC: When to Reason and When to Act? Self Divide-and-Conquer for Compositional Unknown Questions” on OpenResearch

About this paper

Hongru Wang, Boyang Xue, Baohang Zhou, Tianhua Zhang, Cunxiang Wang, Huimin Wang, Guanhua Chen, Kam-fai Wong

The Self Divide-and-Conquer (Self-DC) framework allows Large Language Models to adaptively choose between internal reasoning, external retrieval, or question decomposition, guided by self-assessed confidence scores. This approach enables LLMs to achieve comparable or superior performance on compositional questions while making significantly fewer external retrieval calls.

Key insights

• LLMs can effectively leverage their internal confidence signals to dynamically choose between reasoning and acting, leading to more strategic and efficient problem-solving.
• Decomposition of compositional questions is crucial for improving answer accuracy and reducing retrieval noise by breaking down complex queries into manageable sub-questions.
• The probability-based method for acquiring confidence offers finer granularity and generally leads to better overall performance compared to verbalize-based confidence scores.

Summary by alphaXiv.