Reproduce “mmEgoHand: Egocentric Hand Pose Estimation and Gesture Recognition with Head-mounted Millimeter-wave Radar and IMU” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yizhe Lv, Tingting Zhang, Zhijian Wang, Yunpeng Song, Han Ding, Jinsong Han, Fei Wang
Researchers from Xi'an Jiaotong University and Zhejiang University developed mmEgoHand, an egocentric system using head-mounted millimeter-wave radar and IMU for 3D hand pose estimation and gesture recognition. The system achieved a Mean Per Joint Position Error of 72.73mm for pose estimation and 90.80% accuracy for gesture recognition, outperforming prior mmWave methods and demonstrating robustness to head movements and occlusions.
Key insights
- Fusing IMU data with mmWave radar signals effectively compensates for head-motion artifacts in egocentric sensing, leading to a 24.6% reduction in Mean Per Joint Position Error (MPJPE) compared to using mmWave alone.
- The temporal Context Decoder significantly enhances hand pose estimation by explicitly modeling sequential dependencies across frames, which is crucial for dynamic egocentric scenarios.
- Utilizing estimated 3D hand poses as an intermediate representation substantially improves downstream gesture recognition performance by 20-30 percentage points over direct classification from raw sensor data.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.