Reproduce “MUA: Mobile Ultra-detailed Animatable Avatars” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Heming Zhu, Guoxing Sun, Marc Habermann
MUA (Mobile Ultra-detailed Animatable Avatars) develops a compact representation for animatable full-body digital humans, allowing ultra-high visual fidelity to be rendered in real-time on mobile devices. It achieves over a 1000x reduction in computational cost and a 10x smaller model size compared to server-based teacher models, running at 24 FPS natively on a Meta Quest 3 headset.
Key insights
- Decomposing Gaussian splat textures into wavelet subbands enables a multi-level approach, where different frequency bands can be represented with tailored, more efficient structures (e.g., 2D vs. 1D factorized blendshapes).
- Directly modeling full-resolution Gaussian splat textures with PCA or single-level factorization is computationally prohibitive and insufficient for jointly representing detailed geometry and appearance.
- The blendshape formulation, when applied to appropriate frequency bands (low- and mid-frequency wavelet subbands), proves more effective than 2D convolutional networks for capturing dynamic details under tight computational budgets.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.