Reproduce “Chorus: Multi-Teacher Pretraining for Holistic 3D Gaussian Scene Encoding” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yue Li, Qi Ma, Runyi Yang, Mengjiao Ma, Bin Ren, Nikola Popovic, Nicu Sebe, Theo Gevers and 3 more
Researchers from the University of Amsterdam, ETH Zürich, INSAIT, and the University of Trento developed Chorus, a multi-teacher pretraining framework that distills diverse signals from 2D foundation models into a 3D Gaussian Splatting encoder. This framework yields state-of-the-art performance across open-vocabulary 3D semantic and instance segmentation, visual question answering, and point-cloud tasks, demonstrating improved data efficiency and transferability.
Key insights
- Distilling diverse signals from multiple complementary 2D foundation models (language-aligned, generalist, object-aware) into a 3DGS encoder creates a more holistic and versatile 3D feature representation than single-teacher approaches.
- Features learned directly from 3DGS primitives are highly transferable, enabling state-of-the-art performance not only on 3DGS-native tasks but also when adapted to point-cloud modalities.
- A render-and-distill adaptation strategy significantly simplifies out-of-domain adaptation by leveraging 3DGS's rendering capabilities for online, image-plane supervision, reducing the need for costly 3D pseudo-label precomputation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.