Reproduce “Gaussian Pixel Codec Avatars: A Hybrid Representation for Efficient Rendering” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Divam Gupta, Anuj Pahuja, Nemanja Bartolovic, Tomas Simon, Forrest Iandola, Giljoo Nam
Researchers at Meta Codec Avatars Lab developed Gaussian Pixel Codec Avatars (GPiCA), a hybrid representation for animatable head avatars that combines a semi-transparent mesh with a limited number of 3D Gaussians. This approach delivers photorealism comparable to dense 3D Gaussian Splatting while enabling efficient real-time rendering on mobile VR hardware, achieving up to an 8x reduction in Gaussian count for similar perceptual quality and nearly 2x faster rendering than pure 3DGS models of comparable visual fidelity.
Key insights
- Treating the base mesh as a semi-transparent layer within the 3D Gaussian Splatting rendering paradigm enables the Gaussians to effectively model complex volumetric details in all depths relative to the mesh, outperforming opaque mesh approaches.
- Concentrating a limited number of 3D Gaussians in visually critical, non-surface regions like hair and eyelashes through a static sampling mask maximizes their impact on realism and efficiency.
- A unified, end-to-end differentiable pipeline for hybrid mesh-Gaussian rendering allows the system to optimally learn which representation contributes most to different visual elements for superior photorealism on low-end hardware.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.