Reproduce “Large-scale Codec Avatars: The Unreasonable Effectiveness of Large-scale Avatar Pretraining” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Junxuan Li, Rawal Khirodkar, Chengan He, Zhongshi Jiang, Giljoo Nam, Lingchen Yang, Jihyun Lee, Egor Zakharov and 32 more
LCA reconstructs animatable 3D avatars from a handful of images by pretraining on one million in-the-wild videos, then fine-tuning on multi-view studio captures. It uses 3D Gaussians to represent appearance and pose. On four-image in-the-wild tests, it achieves 28.175 dB PSNR, versus 27.998 dB for mixed-data training; this supports the tested recipe on reconstruction metrics, not causation beyond this comparison.
Key insights
- The pretrain-then-post-train strategy resolves the fidelity-generalization trade-off, consistently outperforming models trained exclusively on studio data, in-the-wild data, or a direct mixture of both.
- Large-scale pretraining on in-the-wild data is crucial for establishing robust human priors, which enable broad generalization to diverse appearances and unseen conditions.
- Post-training on high-quality studio data effectively specializes the pretrained model for precise control, photorealism, and 3D completeness, while enabling modular extensions for features like loose garment support and relighting.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.