Reproduce “WinDEX: Metric 3D Hand Motion Reconstruction in the Wild for Dexterous Teleoperation and Policy Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Hongyu Ma, Hairong Qu, Shiqi Zhao, Yongsong Yang, Kai Shi, Jiahao Ai, Xin You, Peng Yin

WinDEX reconstructs hand motion in a world frame from wearable stereo video and inertial data, using reviewed image keypoints, geometric and temporal fitting, and visual–inertial SLAM. It trains from filtered annotations and supplies motion references for robot policies. On the stated world-frame benchmark, it reports 37.29 mm W-MPJPE versus 97.65 mm for fine-tuned HaWoR; this full-system comparison is protocol-specific.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.