Reproduce “M2StyleGS: Multi-Modality 3D Style Transfer with Gaussian Splatting” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xingyu Miao, Xueqi Qiu, Haoran Duan, Yawen Huang, Xian Wu, Jingjing Deng, Yang Long
M2StyleGS introduces a real-time 3D style transfer method utilizing 3D Gaussian Splatting that accepts multi-modal inputs, addressing issues of abnormal colors and blurred textures in stylized 3D scenes. The approach achieves more precise stylization, reducing short-range LPIPS by up to 32.92% compared to prior methods.
Key insights
- Direct projection methods with MSE loss are insufficient for robustly aligning non-Gaussian distributed CLIP features with VGG style features, leading to visual artifacts and style mismatches.
- Reframing feature alignment as an ODE-guided flow-matching problem, divided into iterative 'subdivisive' steps, allows for a more precise and consistent transformation of feature distributions.
- Integrating auxiliary losses that consider both 2D stylistic priors and global scene-level color regularization is crucial for generating high-quality, multi-view consistent stylized 3D scenes.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.