Reproduce “A Causality-aware Paradigm for Evaluating Creativity of Multimodal Large Language Models” on OpenResearch

About this paper

Zhongzhan Huang, Shanshan Zhong, Pan Zhou, Shanghua Gao, Marinka Zitnik, Liang Lin

This research introduces LoTbench, a causality-aware framework for evaluating the "Leap-of-Thought" (LoT) creativity in Multimodal Large Language Models, alongside CLoTv2, a method designed to enhance these creative abilities. The work demonstrates that LoTbench results strongly correlate with cognitive benchmarks, and CLoTv2 improves MLLMs' capacity to generate human-preferred creative responses on a diverse, multimodal humor dataset.

Key insights

• Chain-of-Thought (CoT) reasoning frameworks do not enhance, and can even impair, Leap-of-Thought (LoT) performance, indicating that creativity requires a distinct cognitive mechanism from sequential logic.
• LoTbench results exhibit a strong positive correlation (Pearson 0.801) with the MMMU multimodal cognition benchmark, suggesting that robust cognitive abilities form a critical foundation for creativity in MLLMs.
• MLLMs can be effectively trained to acquire and enhance LoT abilities using methods like CLoTv2, which improves their capacity to produce human-preferred creative content.

Summary by alphaXiv.