Reproduce “T-3DGS: Removing Transient Objects for 3D Scene Reconstruction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Alexander Markin, Vadim Pryadilshchikov, Artem Komarichev, Ruslan Rakhimov, Peter Wonka, Evgeny Burnaev
T-3DGS, a framework for 3D scene reconstruction, effectively removes transient and semi-transient objects from unconstrained video sequences using 3D Gaussian Splatting. It achieves higher fidelity static scene reconstructions with improved PSNR, SSIM, and LPIPS scores on benchmark datasets, including a newly introduced challenging dataset, by employing unsupervised uncertainty prediction and temporal mask refinement.
Key insights
- Effective unsupervised transient detection can be achieved by leveraging semantic feature consistency differences and a multivariate uncertainty model with KL divergence between real and rendered images, improving robustness over RGB residuals.
- Robustly handling semi-transient objects requires a multi-stage temporal mask propagation strategy that combines spatial refinement and consistent tracking across varying frame rates.
- Mask dilation around transient objects, rather than exact masks, consistently improves reconstruction quality and aids in the training stability of the uncertainty predictor.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.