Reproduce “ST-$π$: Structured SpatioTemporal VLA for Robotic Manipulation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Chuanhao Ma, Hanyu Zhou, Shihan Peng, Yan Li, Tao Gu, Luxin Yan

Researchers developed ST-π, a framework for robotic manipulation that explicitly structures spatiotemporal reasoning at both chunk-level task decomposition and step-level action generation. This approach enabled higher success rates and faster completion times on complex, long-horizon tasks across simulation benchmarks like LIBERO and real-world scenarios using the new STAR dataset.

Key insights

  • Explicitly structuring spatiotemporal modeling at both the planning (chunk) and execution (step) levels significantly enhances stability and coherence for long-horizon robotic manipulation.
  • Integrating distinct spatial and temporal motion generators with complementary attention mechanisms (bidirectional for spatial, causal for temporal) effectively produces both spatially smooth and temporally stable trajectories.
  • Utilizing full 4D observations (multi-frame images + geometric features) is crucial for comprehensive spatiotemporal perception, outperforming simpler 3D or 2D modalities.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.