Reproduce “Temporal Representation Alignment: Successor Features Enable Emergent Compositionality in Robot Instruction Following” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Vivek Myers, Bill Chunyuan Zheng, Anca Dragan, Kuan Fang, Sergey Levine

This work introduces Temporal Representation Alignment (TRA), a method for robot instruction following that enables emergent compositional generalization. TRA uses an auxiliary temporal alignment loss to structure state, goal, and language representations, allowing robots to perform novel, multi-step tasks without explicit planning and achieving substantial performance gains on out-of-distribution compositional tasks in both real-world and simulated environments.

Key insights

  • Emergent compositionality in robotics can arise from structuring representation spaces through temporal and multimodal alignment, rather than solely relying on explicit planning or value functions.
  • An auxiliary objective enforcing temporal consistency between state representations, akin to successor features, is sufficient to enable generalization to novel compositional tasks.
  • Combining goal-conditioned and language-conditioned learning with representation alignment allows for robust instruction following for complex, long-horizon tasks, even with infrequently observed skills.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.