Reproduce “FB-4D: Spatial-Temporal Coherent Dynamic 3D Content Generation with Feature Banks” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jinwei Li, Huan-ang Gao, Wenyi Li, Haohan Chi, Chenyu Liu, Chenxi Du, Yiqian Liu, Mingju Gao and 8 more
FB-4D presents a training-free framework for dynamic 3D (4D) content generation that addresses spatial and temporal inconsistencies by leveraging a Feature Bank mechanism within pre-trained diffusion models. The approach combines dynamic feature storage, a compact representation strategy, and a progressive multi-iteration generation process to achieve an FVD of 724.26, outperforming previous training-free methods and reaching performance competitive with training-based solutions.
Key insights
- Pre-trained diffusion models inherently contain rich feature correspondences that can be leveraged to maintain spatial and temporal consistency in dynamic 3D content generation.
- A dynamic merging strategy for feature banks can compactly represent historical information, preventing memory overflow and error accumulation during progressive generation.
- Autoregressive refinement for 4D content is viable when coupled with intelligent viewpoint selection and cross-iteration feature integration, allowing for iterative quality improvement.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.