Reproduce “Gaussian-JEPA: Joint-Embedding Predictive Learning for 3D Gaussian Splats” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Bin Ren, Qi Ma, Yue Li, Zongyan Han, Yidi Li, Yuqian Fu, Rao Muhammad Anwer, Theo Gevers and 2 more
Gaussian-JEPA predicts latent features for spatially held-out blocks of 3D Gaussian objects rather than reconstructing sampled attributes. This targets variation caused by fixed-budget sampling. Against matched Gaussian-MAE pretraining, its frozen features reduce normalized embedding drift from 0.3400 to 0.2594 while retrieval remains comparable at 92.95% versus 93.00%. These object-level results do not establish equivalence across independently optimized assets or generalization to large scenes.
Key insights
- Predicting high-level latent representations of missing 3D Gaussian Splatting content is more effective and robust than reconstructing raw Gaussian attributes for learning object-level features.
- Employing complementary latent spaces with feature-space grounding helps to learn distinct and informative feature projections (e.g., geometric and appearance-related) without explicit attribute decoding or reliance on negative pairs.
- Training with multi-scale, non-overlapping target blocks exposes the model to various granularities of missing content, leading to more generalized and robust object representations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.