Reproduce “Neural Field Movement Primitives for Joint Modelling of Scenes and Motions” on OpenResearch

About this paper

Ahmet Tekden, Marc Peter Deisenroth, Yasemin Bekiroglu

Neural Field Movement Primitives for Joint Modelling of Scenes and Motions presents a data-efficient learning-from-demonstration framework that uses a shared latent embedding to reconstruct a scene and generate its associated robot motion. Across simulated and real manipulation tasks, the method interpolates to unseen scene configurations, handles multiple valid trajectories and 3D geometry, and reports up to 100% success on the evaluated real-world test cases while using only a small number of demonstrations.

Key insights

• Jointly reconstructing the scene and its motion lets the model infer motion-relevant latent structure directly from observations, reducing reliance on task-specific perception and parameter engineering.
• Low-frequency deformation fields, coarse-to-fine positional encoding, and regularization encourage smooth interpolation while limiting ambiguous or unnecessarily complex scene deformations.
• The same framework supports RGB images, signed distance functions, explicit trajectories, and implicit representations of alternative motion modes.

Summary by alphaXiv.