Reproduce “End-to-End Learning of Geometry and Context for Deep Stereo Regression” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Alex Kendall, Hayk Martirosyan, Saumitro Dasgupta, Peter Henry, Ryan Kennedy, Abraham Bachrach, Adam Bry

The GC-Net, developed by Skydio Inc., introduces an end-to-end deep learning framework for stereo disparity estimation by integrating 3D convolutions for contextual regularization over a cost volume and a differentiable soft argmin for direct sub-pixel regression. This approach achieved state-of-the-art results on the KITTI 2012 and 2015 benchmarks, with relative error reductions of up to 22% over previous methods.

Key insights

  • 3D convolutions are highly effective for learning deep contextual regularization over the cost volume, enabling the network to resolve ambiguities in challenging regions and effectively smooth disparity while preserving details.
  • Formulating disparity estimation as a regression problem with a differentiable soft argmin allows for direct prediction of sub-pixel disparity values, outperforming classification-based approaches and simplifying the overall pipeline.
  • Explicitly constructing a full cost volume and applying multi-scale 3D convolutions for context (rather than simpler approximations) is crucial for achieving superior accuracy and leveraging wide contextual information.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.