Reproduce “Mon3tr: Monocular 3D Telepresence with Pre-built Gaussian Avatars as Amortization” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Fangyu Lin, Yingdong Hu, Zhening Liu, Yufan Zhuang, Zehong Lin, Jun Zhang

Mon3tr presents a monocular 3D telepresence framework that leverages pre-built Gaussian avatars and an amortized computation strategy to achieve photorealistic full-body holographic representations. The system operates on consumer-grade hardware with an ultra-low bandwidth of 0.16 Mbps and an end-to-end latency of 73.1 ms, while maintaining approximately 60 FPS.

Key insights

  • An amortized computation paradigm decouples the heavy avatar reconstruction load from the online interaction, enabling real-time, lightweight data transmission for telepresence.
  • Combining a hybrid parametric human model (SPMM3) with learned mesh deformation and distributed local Gaussian attribute refinement effectively captures fine geometric and appearance dynamics.
  • Transmitting concise semantic parameters allows for ultra-low bandwidth communication, as the high-fidelity avatar is rendered locally at the receiver, leveraging its detailed pre-built representation.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.