Reproduce “STEP: Spatiotemporal Training with Explicit Prediction for Perceptive Humanoid Locomotion” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Youhao Lyu, Chenghao Sun, Zhenxuan Ding, Shuting Zhang, Ziyang Dong, Wenhao Cui
STEP trains humanoid locomotion by jointly learning a compact representation of depth and body motion, alongside a prediction of how that representation changes after an action. Four queries fuse the modalities, while masked cross-modal and action-conditioned prediction objectives train the shared encoder. On hardware, STEP completed 98 of 100 trials, including 20/20 backward stair trials; both failures occurred on gaps.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.