Reproduce “ReDepth Anything: Test-Time Depth Refinement via Self-Supervised Re-lighting” on OpenResearch

About this paper

Ananta R. Bhattarai, Helge Rhodin

Re-Depth Anything refines monocular depth predictions from foundation models at test-time using a self-supervised re-lighting method that leverages 2D diffusion models. This approach improves depth accuracy across diverse datasets, yielding up to an 8.51% reduction in RMSE log on KITTI and a 15% improvement in normal errors on CO3D when applied to Depth Anything 3.

Key insights

• A novel re-synthesis method, leveraging re-lighting to provide a geometry-aware supervisory signal, bypasses the need for physically accurate photometric reconstruction or explicit albedo estimation.
• Targeted optimization of specific network components, rather than direct pixel optimization or full fine-tuning, is crucial for stable and effective test-time depth refinement.
• 2D diffusion models, when used as a realism prior for re-lit images, can effectively guide geometric refinement and correct biases in initial depth predictions.

Summary by alphaXiv.