Reproduce “Gaussian Splatting on the Move: Blur and Rolling Shutter Compensation for Natural Camera Motion” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Otto Seiskari, Jerry Ylilammi, Valtteri Kaatrasalo, Pekka Rantalankila, Matias Turkulainen, Juho Kannala, Esa Rahtu, Arno Solin
This work enhances 3D Gaussian Splatting (3DGS) by integrating a physical image formation model directly into the differentiable rendering pipeline to compensate for motion blur and rolling shutter distortion. The approach leverages Visual-Inertial Odometry (VIO) to regularize pose and velocity optimization, yielding significantly sharper and more accurate 3D reconstructions from casually captured handheld video data, outperforming prior 2D and 3D compensation techniques.
Key insights
- Explicitly modeling physical image formation processes (motion blur, rolling shutter) within a differentiable 3D representation like 3D Gaussian Splatting (3DGS) is more effective for recovering sharp 3D scenes than 2D image pre-processing.
- Integrating velocity estimates from Visual-Inertial Odometry (VIO) as priors, along with optimizing camera poses and velocities, significantly regularizes the ill-posed problem of deblurring and distortion correction.
- A screen-space approximation for motion-induced Gaussian deformation allows for computationally efficient integration of blur and rolling shutter compensation into the 3DGS pipeline.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.