Reproduce “DermDepth: Toward Monocular Metric Scale 3D Reconstruction Models for Dermatology” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Héctor Carrión, Narges Norouzi
The DermDepth model reconstructs metric-scale 3D information from single 2D dermatological images by fine-tuning foundation models with a novel synthetic dataset, D-Synth. This approach significantly improves metric depth accuracy and surface normal estimation while reducing performance disparities across Fitzpatrick skin tones.
Key insights
- State-of-the-art monocular depth foundation models exhibit systematic metric scale errors on dermatological images, overestimating or underestimating depth by factors up to 16x, despite maintaining good relative geometry.
- A synthetic dataset (D-Synth) with pixel-perfect 3D ground truth effectively teaches metric scale and normal prediction, enabling substantial initial scale correction on real dermoscopic data.
- A focused fine-tuning strategy, involving only the scale and normal output heads of the foundation model, proves efficient and prevents overfitting when adapting to limited, noisy real-world dermatological 3D data.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.