Reproduce “DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning” on OpenResearch

About this paper

Ziwei Zheng, Michael Yang, Jack Hong, Chenxiao Zhao, Guohai Xu, Le Yang, Chao Shen, Xing Yu

DeepEyes lets vision-language models interleave text reasoning with image crops selected during the reasoning process, training the full trajectory with reinforcement learning. The crop returns as new visual evidence, without a cold-start on reasoning traces or a separate specialist model. On V* Bench, it scores 90.1% versus 71.2% for reproduced Qwen2.5-VL-7B; same-data comparisons are mixed across benchmarks.

Key insights

• The ability to 'think with images' can emerge natively through end-to-end reinforcement learning, effectively cultivating sophisticated visual reasoning without relying on pre-collected reasoning data for supervised fine-tuning or external specialized models.
• A conditional reward mechanism, which specifically grants a bonus for active perception only when the final answer is correct, is essential for incentivizing effective, perception-aware reasoning and discouraging unnecessary visual actions.
• DeepEyes learns to strategically integrate active perception, evolving through distinct stages from initial exploration to high-frequency engagement and finally to efficient, targeted visual exploitation, demonstrating human-like patterns such as visual search, comparison, and confirmation.

Summary by alphaXiv.