Reproduce “Reconstruction-Augmented Visuomotor Pre-Training of 3D-Grounded Bimanual Dexterity from RGB-Only Human Videos” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Xingyi He, Zhiwen Qiu, Adhitya Polavaram, Yunhao Cao, Cory Fan, Julius Samwer, Saleh Kalantari, Kuan Fang

SpatialDex trains a bimanual policy from human videos by aligning hand motion with reconstructed scene geometry, using geometry as input and a future prediction target, then fine-tuning with robot demonstrations. In matched six-task simulations, it reaches 73.1% mean success versus 58.9% for motion-only pre-training. Across nine simulated tasks, it reaches 76.3%, but still requires depth at deployment and embodiment-specific demonstrations.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.