Reproduce “V4d: voxel for 4d novel view synthesis” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Wanshui Gan, Hongbin Xu, Yi Huang, Shifeng Chen, Naoto Yokoya
The V4D framework introduces a voxel-based approach for 4D novel view synthesis in dynamic scenes from single-view video, addressing limitations of MLP-based methods. It achieves improved rendering quality with a PSNR of 33.72 on synthetic dynamic datasets and significantly faster inference at 0.48 seconds, utilizing novel modules for detail enhancement and color refinement without relying on auxiliary data.
Key insights
- Separating density and texture into distinct voxel grids helps avoid optimization conflicts and enhances the representation capacity for complex dynamic scenes.
- Time-dependent phase shifts within positional encoding effectively re-introduce high-frequency information, counteracting over-smoothing caused by regularization.
- A plug-and-play LUTs refinement module provides consistent pixel-level color improvements with minimal computational overhead, applicable to both 3D static and 4D dynamic synthesis.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.