Reproduce “MoMo: Dial Motion Mode in Robot Manipulation with Spatiotemporal Action Tokenization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yuhan Hu, Hugues Thomas, Peide Huang, Mouli Sivapurapu, Benoit Landry, Arto Kivila
MoMo, a two-stage imitation learning framework developed at Apple, disentangles spatial and temporal action components to enable continuous control over "motion mode" in robot manipulation. The framework allows compositional generalization, successfully applying learned motion styles to tasks never demonstrated in those specific modes while maintaining task completion and generating distinct, interpretable variations in robot dynamics and end-effector orientation.
Key insights
- Explicitly decomposing robot actions into separate spatial and temporal components facilitates the disentanglement of task-specific structure from reusable behavioral motion characteristics.
- Temporal action codes primarily encode motion mode qualities (e.g., speed, acceleration patterns) and are largely task-agnostic, enabling their transfer across different manipulation tasks.
- Spatial action codes predominantly encode task-dependent geometric structures and remain relatively insensitive to variations in motion mode.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.