Reproduce “Imagine3D-LLM: Teaching MLLMs to Imagine 3D Scenes Before Answering” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jaewoo Jung, Hyeonseo Yu, Honggyu An, Jisang Han, Mungyeom Kim, Minkyeong Jeon, Heeseong Shin, Wonjun Moon and 5 more
Imagine3D-LLM trains a multimodal language model to summarize views as a 3D Gaussian Splatting scene before answering questions. Summary tokens are trained to reconstruct views and predict answers, with teacher distillation only during training. On SPAR-Bench, Imagine3D-LLM-7B scores 68.5 overall versus 60.9 for a matched baseline, but the method remains limited to indoor scenes and fixed capacity.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.