Reproduce “Vision-Language Models Can Self-Improve Reasoning via Reflection” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Kanzhi Cheng, Yantao Li, Fangzhi Xu, Jianbing Zhang, Hao Zhou, Yang Liu

This work introduces R3V, a self-training framework that enables Vision-Language Models (VLMs) to enhance their multimodal reasoning by iteratively generating and reflecting on Chain-of-Thought (CoT) rationales. R3V improves reasoning performance by an average relative accuracy of 32.8% on Qwen-VL, effectively transforming weak CoT capabilities into strong reasoning skills across various benchmarks.

Key insights

  • MLLMs can effectively self-improve their reasoning capabilities through an iterative bootstrapping process of self-generated CoT rationales, reducing reliance on costly human or proprietary model annotations.
  • Learning from both correct (positive) and incorrect (negative) reasoning paths is crucial for robust multimodal self-improvement, especially through mechanisms like `self-refine` and `self-select`.
  • A test-time selection mechanism, trained via `self-select`, can further boost inference performance by allowing the model to compare multiple reasoning paths and choose the most accurate answer, even generalizing to out-of-distribution tasks.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.