Reproduce “Compositional Motion Generation from Demonstration with Object-Centric Neural Fields” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Ahmet Ercan Tekden, Yasemin Bekiroglu

A generative learning-from-demonstration framework enables compositional robotic behavior by integrating object-centric neural fields for scene modeling with a temporal mixture-of-experts for motion generation, conditioned on shared object-level latent representations. This approach achieves superior data efficiency in learning complex manipulation tasks from few demonstrations, demonstrating robustness and generalization on simulation benchmarks and real-robot tasks, such as 25/25 successful pick-and-place and 10/10 successful box-in-drawer with only four training examples.

Key insights

  • Compositional modeling at both the perceptual (object-centric neural fields) and action (temporal MoE) levels, linked by shared latent representations, significantly enhances data efficiency and generalization in LfD.
  • Learning smooth, interpretable object-centric scene representations directly from few images provides a robust visual grounding for motion generation without requiring extensive manual feature engineering.
  • Leveraging temporal gating in motion generation allows for sequential compositionality, where different phases of a task are conditioned on distinct subsets of object-specific latent variables.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.