Reproduce “DeFM: Learning Foundation Representations from Depth for Robotics” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Manthan Patel, Jonas Frey, Mayank Mittal, Fan Yang, Alexander Hansson, Amir Bar, Cesar Cadena, Marco Hutter

DeFM pretrains reusable depth-image encoders through a depth-specific adaptation of DINOv2 self-distillation, using 60.4 million images from 18 datasets and scale-aware inputs. Its frozen encoders keep weights fixed while downstream policies or probes are learned. In simulated grasping, DeFM reaches 0.8089 of teacher performance versus 0.6894 for a VAE baseline; transfer is shown qualitatively for real-world navigation only.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.