Reproduce “Beyond Perceptual Shortcuts: Causal-Inspired Debiasing Optimization for Generalizable Video Reasoning in Lightweight MLLMs” on OpenResearch

About this paper

Jingze Wu, Quan Zhang, Hongfei Suo, Zeqiang Cai, Hongbo Chen

A causal-inspired debiasing optimization framework, VideoThinker, enables lightweight Multimodal Large Language Models (MLLMs) to achieve robust, generalizable video reasoning by actively countering 'perceptual shortcuts' in training data. This 3B parameter model established new state-of-the-art performance for its size, outperforming some larger models and demonstrating significant accuracy improvements on causal inference benchmarks.

Key insights

• 'Perceptual shortcuts' in training data are a primary cause of reasoning degradation in lightweight MLLMs during RL fine-tuning, as these models exploit superficial correlations instead of developing genuine understanding.
• A Structural Causal Model revealed that standard training methods, including RL, inadvertently encourage models to rely on an efficient 'spurious shortcut pathway' (Q O A) due to input query confounding.
• Actively repelling a policy from a specialized 'bias model' that embodies shortcut behaviors is an effective causal-inspired intervention to compel the main model to learn robust, generalizable reasoning pathways.

Summary by alphaXiv.