Reproduce “Learning to Recover 3D Scene Shape from a Single Image” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Wei Yin, Jianming Zhang, Oliver Wang, Simon Niklaus, Long Mai, Simon Chen, Chunhua Shen
Researchers at Adobe Research and The University of Adelaide introduce a two-stage, fully data-driven framework for reconstructing accurate 3D scene shapes from single monocular images by explicitly recovering depth shift and camera focal length. This approach leverages 3D point cloud networks for parameter estimation and achieves new state-of-the-art zero-shot depth prediction performance, significantly reducing 3D reconstruction errors compared to prior methods.
Key insights
- Leveraging 3D point cloud networks is more effective and generalizable for recovering camera and depth parameters like shift and focal length than 2D image-based methods, due to a smaller domain gap in 3D space.
- Explicitly correcting for depth shift and focal length is essential for transforming relative, scale-shift invariant depth maps into geometrically accurate and undistorted 3D scene reconstructions.
- Novel loss functions, such as Image-level Normalized Regression and Pair-wise Normal loss, significantly enhance the robustness and local geometric quality of monocular depth predictions, especially when trained on heterogeneous datasets.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.