Reproduce “ThinkWise: Learning When to Think in Large Multimodal Reasoning Models via Visual and Difficulty Perception” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Wenkang Zhou, Chen Yang, Jiaheng Peng, Xitong Li, Hongzhi Yin, Yunke Zhang, Lu Yu, Peng-Fei Zhang and 2 more

ThinkWise trains multimodal models to condition reasoning-length rewards on visual engagement, correctness, and question difficulty. It estimates engagement from image-token attention relative to other responses to the same question, adjusting training rewards rather than inference-time stopping. On MMK12, Qwen2.5-VL-3B reaches 67.15% accuracy with 342.6 average response length; visual engagement is only a proxy, and the length unit is unspecified.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.