Reproduce “SemanticMoments: Training-Free Motion Similarity via Third Moment Features” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Saar Huberman, Kfir Goldberg, Or Patashnik, Sagie Benaim, Ron Mokady
Researchers from BRIA AI and Tel Aviv University introduce SemanticMoments, a training-free method that generates video representations focusing on semantic motion similarity by analyzing higher-order temporal moments of patch-wise features from pretrained semantic models. This approach mitigates appearance bias, achieving 86.40% retrieval accuracy on the SimMotion-Synthetic benchmark and 42.50% on the SimMotion-Real benchmark, outperforming other video understanding techniques.
Key insights
- Video understanding models exhibit a pervasive appearance bias, leading to failures in motion-centric retrieval when visual style or identity changes.
- Higher-order temporal statistical moments, specifically variance and skewness, provide complementary information beyond the mean, effectively capturing subtle semantic motion dynamics.
- The use of patch-level feature granularity combined with temporal moments is crucial for disentangling motion from appearance and preserving fine motion structure.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.