Reproduce “Exploring Surround-View Fisheye Camera 3D Object Detection” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Changcai Li, Wenwei Lin, Zuoxun Hou, Gang Chen, Wei Zhang, Huihui Zhou, Weishi Zheng

Researchers from Sun Yat-sen University and Pengcheng Laboratory introduced the Fisheye3DOD dataset and two tailored frameworks, FisheyeBEVDet and FisheyePETR, to systematically evaluate and improve 3D object detection using surround-view fisheye cameras. Their methods, which incorporate spherical modeling, achieved FDS improvements of 4.5 and 6.2 points respectively over rectification baselines, while also demonstrating the inherent robustness of fisheye systems to sensor failures and their suitability for near-field perception.

Key insights

  • Applying pinhole-based 3D object detectors to rectified fisheye images results in a significant accuracy drop (over 12 FDS points) due to irreversible pixel compression and information loss.
  • Directly modeling the spherical geometry of fisheye cameras at the feature level, specifically using an equirectangular representation, is substantially more effective than image-level rectification for 3D object detection.
  • Multi-fisheye camera systems exhibit inherent robustness to sensor failures and excel in near-field object detection, making them suitable for low-speed autonomous applications.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.