Reproduce “TIGaussian: Disentangle Gaussians for Spatial-Awared Text-Image-3D Alignment” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiarun Liu, Qifeng Chen, Yiru Zhao, Minghua Liu, Baorui Ma, Sheng Yang
TIGAUSSIAN proposes a tri-modal framework for text-image-3D alignment that disentangles 3D Gaussian Splatting attributes and enhances 3D perception using a diffusion-enhanced multi-view fusion module. The system achieves state-of-the-art performance across various 3D multi-modal tasks, including zero-shot classification and cross-modal retrieval.
Key insights
- Disentangling the intrinsic attributes of 3D Gaussian Splatting primitives allows for more specialized compression and abstraction of information, leading to richer and less entangled 3D feature representations.
- Leveraging multi-view images generated by diffusion models, combined with a fusion mechanism, provides a more comprehensive 3D-aware visual context, overcoming the limitations of single-view alignment.
- A dedicated projection module using learnable queries effectively aligns 3D features into a latent space compatible with text embeddings, improving the correlation between 3D and textual modalities.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.