Reproduce “Interactive3D: Create What You Want by Interactive 3D Generation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Shaocong Dong, Lihe Ding, Zhanpeng Huang, Zibin Wang, Tianfan Xue, Dan Xu
Researchers from HKUST, CUHK, SenseTime Research, and Shanghai AI Lab developed Interactive3D, a framework that enables precise user control during 3D object generation using a two-stage approach leveraging Gaussian Splatting and InstantNGP. The system facilitates interactive manipulation with features like deformable dragging and local semantic editing, yielding a CLIP R-Precision of 0.94 and generating objects in an average of 50 minutes.
Key insights
- Combining explicit (Gaussian Splatting) and implicit (InstantNGP) 3D representations in a cascading manner effectively balances user interactivity and high-fidelity output quality.
- Direct manipulation of explicit 3D representations, such as Gaussian blobs, allows for intuitive control over geometric and semantic features during the generation process.
- Localizing optimization and refinement using techniques like adaptive camera zoom-in for SDS and part-specific hash tables in InstantNGP is crucial for efficient and precise interactive editing.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.