Reproduce “SimpleRecon: 3D Reconstruction Without 3D Convolutions” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Mohamed Sayed, John Gibson, Jamie Watson, Victor Prisacariu, Michael Firman, Clément Godard

This paper presents SimpleRecon, a 3D reconstruction method that achieves high-quality results by focusing on accurate multi-view depth prediction using a 2D convolutional neural network, thereby avoiding the use of computationally intensive 3D convolutions. SimpleRecon integrates geometric and keyframe metadata into a novel cost volume to boost depth accuracy, leading to state-of-the-art depth estimation on ScanNetv2 and competitive 3D reconstruction quality at a significantly lower latency of 72ms per frame.

Key insights

  • Integrating readily available geometric and keyframe metadata directly into the cost volume significantly enhances depth estimation accuracy without requiring 3D convolutions.
  • Carefully designed 2D CNN architectures for multi-view depth prediction, when combined with off-the-shelf TSDF fusion, can achieve reconstruction quality competitive with methods relying on expensive 3D convolutions.
  • Implicitly encoding metadata, such as by ordering source features based on pose distance, further boosts performance by allowing the network to learn contextual priors.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.