Reproduce “Metis-RISE: RL Incentivizes and SFT Enhances Multimodal Reasoning Model Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Haibo Qiu, Xiaohan Lan, Fanfan Liu, Xiaohu Sun, Delian Ruan, Peng Shi, Lin Ma

Meituan introduces Metis-RISE, a hybrid training framework that first uses Reinforcement Learning to activate latent multimodal reasoning in MLLMs, then applies Supervised Fine-Tuning to refine these abilities. The approach yields state-of-the-art performance on reasoning benchmarks for both 7B and 72B parameter open-source models, demonstrating competitive results against leading proprietary models.

Key insights

  • Initiating training with Reinforcement Learning (RL) is more effective for activating latent reasoning and encouraging exploration in MLLMs than starting with Supervised Fine-Tuning (SFT).
  • A targeted SFT phase, following RL, can specifically address inconsistent reasoning application and fill fundamental knowledge gaps, further enhancing overall model performance.
  • The synergistic combination of RL-first incentivization and problem-specific SFT enhancement leads to cumulative improvements in multimodal reasoning capabilities.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.