Reproduce “Object segmentation from common fate: Motion energy processing enables human-like zero-shot generalization to random dot stimuli” on OpenResearch

About this paper

Matthias Tangemann, Matthias Kümmerer, Matthias Bethge

A study from the University of Tübingen demonstrates that a classical neuroscience-inspired motion energy model exhibits human-level zero-shot generalization for object segmentation from random dot motion, achieving an IoU of 60.0% on such stimuli, which significantly surpasses the performance of state-of-the-art deep optical flow models. This robust perception contrasts with deep learning models, which often show near-chance performance, highlighting their reliance on appearance cues.

Key insights

• State-of-the-art deep optical flow models heavily rely on static appearance cues, leading to a significant failure in zero-shot generalization to appearance-agnostic stimuli.
• The neuroscience-inspired motion energy model intrinsically processes motion through spatio-temporal filtering, enabling robust, appearance-agnostic segmentation capabilities akin to human perception.
• Training or fine-tuning the motion energy model's parameters with gradient descent on standard datasets consistently degrades its zero-shot generalization, suggesting the importance of its original fixed, biologically-plausible filters.

Summary by alphaXiv.