Reproduce “Latent Intrinsics Emerge from Training to Relight” on OpenResearch

About this paper

Xiao Zhang, William Gao, Seemandhar Jain, Michael Maire, David A.Forsyth, Anand Bhattad

A fully data-driven and unsupervised method for image relighting is presented, which disentangles scene characteristics from illumination by encoding them as latent variables. The approach achieves state-of-the-art relighting accuracy and implicitly learns to extract albedo-like maps and control visible luminaires from training on paired images of the same scene under different lighting conditions.

Key insights

• Intrinsic scene properties, such as albedo, can emerge naturally and without explicit supervision from a deep learning model trained solely for the task of image relighting.
• A structural bottleneck, implemented through constrained scaling of extrinsic features during training, is crucial for forcing disentanglement between intrinsic and extrinsic representations.
• The model implicitly learns high-level concepts like controlling visible light sources (luminaires) despite receiving no explicit supervision about their physical properties or states.

Summary by alphaXiv.