Reproduce “Sen-Cap: Sensor-Flexible and Noise-Resilient Human Motion Capture via LiDAR-Camera Integration” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Aoru Xue, Yujing Sun, Yiming Ren, Kwok-Yan Lam, Mao Ye, Yuexin Ma
Sen-Cap, developed by researchers at ShanghaiTech University and Nanyang Technological University, introduces a human motion capture framework that integrates LiDAR and camera data without explicit sensor calibration, achieving robust performance under flexible sensor configurations and severe environmental noise. The framework reduces joint and angular errors compared to existing methods, demonstrating strong generalization across diverse datasets and maintaining accuracy even with partial sensor failure or clutter.
Key insights
- Aligning heterogeneous sensor data to a human-centric coordinate system allows for robust, calibration-free multi-modal fusion, effectively reducing errors and enabling flexible sensor deployments.
- An iterative refinement strategy for point cloud centroids significantly enhances the accuracy and stability of global human translation, making the system resilient to environmental noise and partial occlusions.
- Adaptive sensor fusion, reinforced by sensor dropout training, allows the system to flexibly handle varying numbers and types of LiDAR and camera inputs without requiring retraining.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.