Reproduce “Jigsaw-R1: A Study of Rule-based Visual Reinforcement Learning with Jigsaw Puzzles” on OpenResearch

About this paper

Zifu Wang, Junyi Zhu, Bo Tang, Zhiyu Li, Feiyu Xiong, Jiaqian Yu, Matthew B. Blaschko

Researchers from KU Leuven and collaborators explored rule-based reinforcement learning (RL) for multimodal large language models (MLLMs) using jigsaw puzzles as a structured experimental framework. They found that MLLMs robustly generalize to various visual downstream tasks after training on jigsaw puzzles, with complex reasoning patterns being pre-existing and amplified rather than emerging, and demonstrated RL's superior generalization compared to Supervised Fine-Tuning (SFT).

Key insights

• Complex reasoning patterns in MLLMs are pre-existing and activated or amplified by structured tasks and increased difficulty, rather than suddenly emerging (the "aha moment" is not observed).
• Rule-based RL training offers superior generalization to diverse downstream visual tasks compared to Supervised Fine-Tuning (SFT) for MLLMs.
• The effectiveness of generalization depends on specific puzzle configurations, with 'pair' jigsaw puzzles often yielding better transfer and larger puzzle sizes improving generalization for non-thinking models.

Summary by alphaXiv.