Reproduce “ReasonX: MLLM-Guided Intrinsic Image Decomposition” on OpenResearch

About this paper

Alara Dirik, Tuanfeng Wang, Duygu Ceylan, Stefanos Zafeiriou, Anna Frühstück

ReasonX adapts pretrained intrinsic image decomposition models to real, unlabeled images by using an MLLM as a frozen judge of pairwise depth, albedo, illumination, and surface-orientation relationships. With GRPO fine-tuning on 10,000 COCO images, the resulting PRISM-X model lowers albedo WHDR on IIW from 17.2 to 12.9, reduces depth AbsRel on ETH3D from 0.142 to 0.077, and improves cross-modal consistency, while gains for surface normals remain modest.

Key insights

• Directly training the MLLM for relative comparisons is much more reliable than deriving comparisons from absolute value-bin predictions: held-out synthetic accuracy reaches 96.2% for depth versus 47.5% for the derived alternative.
• Relational rewards improve real-image robustness without requiring dense intrinsic annotations, with the clearest benefits appearing in albedo brightness handling and depth transfer to outdoor-heavy data.
• The reward must be controlled: removing KL regularization degrades performance, while a geometry-only consistency reward does not transfer well to albedo or illumination.

Summary by alphaXiv.