Reproduce “ScenePainter: Semantically Consistent Perpetual 3D Scene Generation with Concept Relation Alignment” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chong Xia, Shengjun Zhang, Fangfu Liu, Chang Liu, Khodchaphun Hirunyaratsameewong, Yueqi Duan
ScenePainter introduces a framework for perpetual 3D scene generation that resolves the semantic drift issue by aligning an outpainter's scene-specific prior with a hierarchical SceneConceptGraph and dynamic refinement. This method generates semantically consistent and visually diverse 3D view sequences, with user studies showing strong preference over previous methods for visual quality, diversity, and consistency.
Key insights
- Encoding multi-level scene concepts and their intricate relations into a hierarchical graph structure (SceneConceptGraph) is crucial for maintaining semantic consistency across diverse 3D generations.
- Combining Textual Inversion and DreamBooth for scene-specific prior customization allows diffusion models to deeply understand and preserve scene identity, preventing semantic drift.
- Dynamically refining the concept graph and leveraging test-time training provides a mechanism for controllable scene evolution and real-time editing while preserving overall coherence.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.