Reproduce “Depth Anything V2” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, Hengshuang Zhao

Depth Anything V2 trains a discriminative monocular depth model by first learning from precise synthetic labels, then using its largest teacher to pseudo-label 62M unlabeled real images for student training. Its ViT-S model reaches 95.3% pairwise accuracy on the curated DA-2K relative-depth benchmark, versus 86.8% for Marigold. This does not establish universal or metric-depth superiority.

Key insights

  • Efficient discriminative models can achieve fine-grained depth details, provided they are trained on precise synthetic data rather than noisy real-labeled images.
  • Synthetic data, while precise, has a domain gap and limited scene coverage, which can be effectively bridged by generating pseudo-labels from a synthetic-trained teacher model on large-scale unlabeled real images.
  • The gradient matching loss is crucial for achieving sharp, fine-grained depth predictions when supervised by precise synthetic labels, an effect not observed with coarse real-labeled data.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.