Reproduce “Real-time Street Human Motion Capture” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yanquan Chen, Fei Yang, Tianyu Lang, Guanfang Dong, Anup Basu
Researchers at the University of Alberta developed a real-time human motion capture system tailored for street scene videos. It leverages a ResNet-34 model for 3D human keypoint extraction, utilizing a heatmap and offset approach, and enables real-time 3D animation of human models within a simulated street environment.
Key insights
- The combination of heatmap and offset outputs from a deep learning model effectively achieves both coarse and precise 3D joint localization from monocular video in visually complex environments.
- Pre-processing video input with blurring using an Xception CNN significantly enhances the robustness of motion capture against challenges posed by color overlap and cluttered backgrounds in street scenes.
- Implementing a low-pass filter on the generated 3D keypoint data is crucial for achieving temporally smooth and stable animated movements, addressing common jittering issues in deep learning-based pose estimation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.