Reproduce “Controllable Dynamic 3D Shape Generation via 3D Trajectories and Text” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jaeyeong Kim, Ines Kim, Jahyeok Koo, Seungryong Kim
Researchers from KAIST AI developed T2Mo, a feed-forward framework for controllable dynamic 3D shape generation that integrates 3D trajectories and text prompts. T2Mo achieved superior prompt alignment, motion expressiveness, and overall quality compared to text-conditioned and cascaded video-based baselines, demonstrating a 5-10 times lower 3D trajectory error.
Key insights
- Combining high-level semantic text with explicit, spatially grounded 3D trajectories enables precise and intuitive control over dynamic 3D shape generation.
- The Shape-Grounded Trajectory Embedding (SGTE) effectively addresses the challenge of arbitrary trajectory input configurations by ensuring localized, geometry-aware conditioning across the mesh.
- A two-stage training scheme with gradual sparsification of trajectory guidance and specific data augmentations is crucial for developing a robust model capable of handling diverse user inputs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.