Reproduce “PFDepth: Heterogeneous Pinhole-Fisheye Joint Depth Estimation via Distortion-aware Gaussian-Splatted Volumetric Fusion” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhiwei Zhang, Ruikai Xu, Weijian Zhang, Zhizhong Zhang, Xin Tan, Jingyu Gong, Yuan Xie, Lizhuang Ma
PFDepth from Shanghai Jiao Tong University and East China Normal University introduces a framework for joint depth estimation using heterogeneous pinhole and fisheye cameras, integrating distortion-aware feature fusion and 3D Gaussian Splatting. It achieves state-of-the-art performance on KITTI-360 and RealHet datasets by effectively leveraging complementary camera information.
Key insights
- Explicitly handling heterogeneous camera types and leveraging their complementary information through the HSF module provides a performance advantage over homogeneous or direct mixing approaches.
- The 3D Gaussian Splatting-based dynamic sampling effectively refines coarse voxel representations, restoring fine-grained structure, especially for thin objects and sparse areas, with minimal computational overhead.
- Consistent aspect ratios across heterogeneous views can further enhance performance, and 3DGS can serve as an efficient bridge to adapt pre-trained 2D models for spatially-aware 3D geometry tasks.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.