Reproduce “Diversity of Thought Improves Reasoning Abilities of LLMs” on OpenResearch

About this paper

Ranjita Naik, Varun Chandrasekaran, Mert Yuksekgonul, Hamid Palangi, Besmira Nushi

DIVERSEPROMPTING enables large language models to generate and integrate diverse reasoning approaches and personas, enhancing their complex reasoning capabilities. This methodology achieves state-of-the-art accuracy on tasks like planning and constraint satisfaction while also improving the accuracy-cost trade-off compared to existing prompting techniques.

Key insights

• LLMs possess a rich, latent array of knowledge and problem-solving strategies that can be effectively elicited and leveraged through systematically tailored prompting.
• Explicitly guiding LLMs with diverse, LLM-generated reasoning approaches and personas is more effective for complex tasks than generic step-by-step instructions or purely stochastic token-level variations.
• The proposed methods push the Pareto frontier for the accuracy-cost trade-off, achieving higher performance at comparable or lower inference costs, especially with the IDIV-SE approach.

Summary by alphaXiv.