Reproduce “Fast Reasoning Segmentation for Images and Videos” on OpenResearch

About this paper

Yiqing Shen, Mathias Unberath

Johns Hopkins University researchers developed FastReasonSeg, an efficient framework enabling multi-step reasoning segmentation on edge devices for both images and videos. The framework leverages a digital twin representation and a two-stage distillation process, with its 0.6B-parameter variant achieving 7.79 FPS and 2.1GB memory consumption while surpassing larger, established models in performance on reasoning segmentation benchmarks.

Key insights

• Structured digital twin representations effectively decouple perception from reasoning, facilitating the transfer of complex reasoning chains during distillation.
• A two-stage distillation approach combining supervised fine-tuning with reinforcement learning, guided by a reasoning-aware reward, is essential for transferring multi-step reasoning capabilities.
• Efficient, compact models can achieve superior performance on complex reasoning segmentation tasks when knowledge from large teacher models is effectively distilled using this framework.

Summary by alphaXiv.