Reproduce “BLARM: Animating 3D Objects from Video via Blending Latent Rigid Motion Primitives” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Pradyumn Goyal, Yizhak Ben-Shabat, Hsueh-Ti Derek Liu, Haomiao Jiang, Snehasish Mukherjee, Kyle Spence, Mark Stauber, Evangelos Kalogerakis and 1 more

BLARM is a feed-forward method that animates 3D meshes from monocular video by factorizing object motion into time-varying rigid components and time-invariant vertex-to-component skinning weights. The system achieves superior geometric accuracy and temporal consistency, demonstrating a 70.8% reduction in absolute trajectory error on ActionBench (CD-3D of 1.71 vs. 2.49 for the next best baseline) and improved temporal realism (better FVD scores) across multiple benchmarks.

Key insights

  • Representing object animation through a compact set of learned, time-varying rigid motion components combined with time-invariant vertex-to-component skinning weights offers a flexible yet structured deformation model.
  • Factorized spatial-temporal attention is an efficient mechanism for capturing both intra-frame component interactions and inter-frame temporal consistency in animation from video.
  • Judiciously designed training objectives, including entropy regularization and motion-aware contrastive learning, can implicitly discover interpretable, part-aware motion structures without direct supervision for rigging.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.