Reproduce “Everybody Tracking Every Body” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Daeyun Shin, Yunhan Zhao, Shu Kong, Alexander C. Berg, Charless Fowlkes

Researchers introduce a conditional diffusion model for multi-person 3D body pose estimation from synchronized egocentric cameras, effectively fusing dense but coarse egocentric head motion with sparse yet precise exocentric visual observations. This method achieves improved accuracy for both global and relative poses, reducing MPJPE to 90.7mm and PA-MPJPE to 45.9mm on the EgoHumans dataset.

Key insights

  • Egocentric head motion and exocentric visual observations provide complementary information: one offers full temporal coverage and absolute positioning, while the other gives more accurate relative poses.
  • Explicitly learning to integrate noisy, intermittent observations within a conditional diffusion framework is more effective than simpler imputation or posterior sampling methods.
  • A strong, learned motion prior, combined with the fusion mechanism, allows for robust pose estimation even when observational data is sparse or entirely missing for periods.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.