Reproduce “Stream3D: Sequential Multi-View 3D Generation via Evidential Memory” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kaichen Zhou, Zeyang Bai, Xinhai Chang, Mengyu Wang, Paul Liang, Fangneng Zhan
Stream3D introduces a training-free streaming mechanism that enables pre-trained 3D generative models to process long monocular video streams, generating temporally consistent 3D outputs with a constant memory footprint. It achieves this by employing an Adaptive Evidential Memory, which selectively retains the most informative historical views at a token level, leading to improved geometry and appearance compared to existing single-view, multi-view, and streaming baselines.
Key insights
- Cross-attention maps within a frozen 3D generator can be leveraged to compute per-token evidence scores, quantifying a view's informativeness and spatial relevance for different parts of the 3D scene.
- Maintaining a compact, token-level 'evidential memory' allows for efficient, long-range context preservation with a constant memory footprint, circumventing the scalability issues of latent state propagation.
- By selecting and weighting conditioning views based on this explicit evidence, the system can achieve temporal consistency and prevent performance degradation over extended sequences, which is challenging for implicit state-transport methods.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.