Reproduce “The Midas Touch for Metric Depth” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yu Ma, Zizhan Guo, Zuyi Xiong, Haoran Zhang, Yi Feng, Hongbo Zhao, Hanli Wang, Rui Fan

The Midas Touch for Depth (MTD) presents a training-free framework that converts relative depth predictions from foundation models into accurate metric depth, leveraging extremely sparse 3D cues. This method achieves state-of-the-art depth completion performance while enabling real-time metric depth estimation on embedded systems.

Key insights

  • Local scale inconsistencies in relative depth can be effectively addressed by segment-wise calibration, where inverse depth serves as an effective scalar proxy for mapping relative to metric depth.
  • A sparse graph optimization scheme can robustly propagate calibration parameters from seeded segments to unseeded ones, enhancing metric accuracy across the entire scene.
  • A discontinuity-aware geodesic cost, informed by depth's second-order derivatives, is crucial for refining depth at the pixel level while preserving object boundaries and preventing erroneous propagation.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.