Reproduce “LAMP: Localization Aware Multi-camera People Tracking in Metric 3D World” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Nan Yang, Julian Straub, Fan Zhang, Richard Newcombe, Jakob Engel, Lingni Ma
LAMP, developed by Meta Reality Labs Research, introduces a framework for real-time 3D multi-person tracking from egocentric multi-camera headsets. It achieves superior world-space localization and coverage by leveraging known device motion to process 2D keypoints as 3D rays, improving tracking accuracy and generalizability with simulated data.
Key insights
- Explicitly leveraging known 6-DoF camera poses to lift 2D observations into a world frame early in the pipeline effectively disentangles observer egomotion from target human motion.
- Representing observations as spatio-temporal 3D ray clouds in a unified world coordinate system enables robust fusion of asynchronous, partial, and multi-view data for consistent tracking.
- Abstracting raw pixels to 3D rays allows for efficient training with simulated data from arbitrary 3D human motion datasets, mitigating data scarcity and improving generalization across different headset camera configurations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.