Reproduce “STEP: Warm-Started Visuomotor Policies with Spatiotemporal Consistency Prediction” on OpenResearch

About this paper

Jinhao Li, Yuxuan Cong, Yingqiao Wang, Hao Xia, Shan Huang, Yijia Zhang, Ningyi Xu, Guohao Dai

STEP (Warm-Started Visuomotor Policies with Spatiotemporal Consistency Prediction) drastically reduces the inference latency of diffusion-based visuomotor policies by integrating spatiotemporal consistency prediction and adaptive perturbation. This approach achieves high task success rates with as few as two denoising steps, enabling real-time robotic control with end-to-end latencies as low as 20ms on real-world tasks.

Key insights

• Explicitly enforcing both spatial (state-conditioned distributional alignment) and temporal (smoothness between consecutive actions) consistency during warm-start is crucial for maintaining action quality and achieving low latency.
• Adaptive perturbation, driven by velocity-aware stagnation detection, is essential for robust real-world robotic deployment, preventing execution deadlocks and ensuring continuous actuation.
• A theoretically grounded warm-start strategy allows diffusion policies to retain their generative expressiveness and high success rates with significantly fewer denoising steps, effectively improving the latency-quality trade-off.

Summary by alphaXiv.