Reproduce “Lift, See, Act: Hierarchical Robot Policy Pretraining with 3D Foundation Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yiyuan Ge, Changxing Ding, Ziyu Hao, Zijie Zheng, Xiangmin Xu
LSA trains a visual encoder from human videos for robot manipulation without using 3D foundation models during inference. It aligns features to geometry and depth, guides attention to interaction regions, and predicts 3D wrist motion before pairing the encoder with a behavior-cloning policy. On four MetaWorld tasks, it reaches 92.2% average success, compared with 40.7% for direct Depth Pro features.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.