Reproduce “Comp4D: LLM-Guided Compositional 4D Scene Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Dejia Xu, Hanwen Liang, Neel P. Bhatt, Hezhen Hu, Hanxue Liang, Konstantinos N. Plataniotis, Zhangyang Wang
A framework generates compositional 4D scenes from text by leveraging LLMs for semantic guidance and trajectory planning, combined with deformable 3D Gaussian Splatting for representation, outperforming prior object-centric methods in visual and video quality metrics and achieving real-time rendering at 70 FPS.
Key insights
- Decomposing compositional 4D scene generation into LLM-guided global motion planning and a generative model's local deformation tasks simplifies complexity and improves scene coherence.
- A flexible optimization strategy that alternates between object-centric and full-scene rendering is crucial for maintaining individual object quality and stable gradients in multi-object 4D scenes.
- Deformable 3D Gaussian Splatting, augmented with explicit physics-based losses, provides an efficient and robust representation for high-fidelity dynamic scenes, enabling real-time rendering performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.