Reproduce “V-RGBX: Video Editing with Accurate Controls over Intrinsic Properties” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ye Fang, Tong Wu, Valentin Deschaintre, Duygu Ceylan, Iliyan Georgiev, Chun-Hao Paul Huang, Yiwei Hu, Xuelin Chen and 1 more
V-RGBX introduces an end-to-end framework for intrinsic-aware video editing, enabling precise and physically grounded manipulation of properties like albedo, normal, material, and irradiance. The system demonstrates superior performance in inverse rendering, intrinsic-conditioned video synthesis, and temporally consistent keyframe-based editing, setting a new standard for controllable video generation.
Key insights
- Explicitly disentangling intrinsic properties within a diffusion model framework enables physically plausible and consistent video edits, overcoming limitations of appearance-level manipulation.
- An interleaved conditioning mechanism provides an effective strategy for integrating sparse user edits across different intrinsic modalities and timeframes into a unified, temporally coherent conditioning input.
- The introduction of temporal-aware intrinsic embeddings within a chunk-based video processing architecture is crucial for disambiguating modality identities over time and improving temporal stability during synthesis.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.