Reproduce “Training-Free Multi-View Extension of IC-Light for Textual Position-Aware Scene Relighting” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiangnan Ye, Jiedong Zhuang, Lianrui Mu, Wenjie Zheng, Jiaqi Hu, Xingze Zou, Jing Wang, Haoji Hu
GS-Light, from Zhejiang University, introduces a training-free framework for text-guided, position-aware relighting of 3D Gaussian Splatting scenes. It integrates Large Vision-Language Models (LVLMs) for lighting prior extraction and an enhanced multi-view consistent diffusion model, achieving superior textual alignment and visual quality while preserving scene structure across viewpoints.
Key insights
- Large Vision-Language Models can be effectively leveraged with geometric estimators to extract precise, position-aware lighting priors from natural language prompts, guiding the relighting process.
- Introducing cross-view attention and normalizing the fundamental matrix significantly enhances numerical stability and consistency for applying 2D diffusion models across multiple views in a 3D context.
- An iterative dataset updating strategy, where 3DGS renders are repeatedly relit and used for fine-tuning, allows 2D image-level edits to converge into a coherent 3D scene representation without extensive retraining.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.