Reproduce “LCCNet: LiDAR and Camera Self-Calibration using Cost Volume Network” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Xudong Lv, Boya Wang, Dong Ye, Shuo Wang

Researchers at Harbin Institute of Technology developed LCCNet, a deep learning framework for real-time LiDAR-camera self-calibration that explicitly models feature correlation using a cost volume network. The system achieves a mean absolute translation error of 0.297 cm and a mean absolute rotation error of 0.017 The system achieves a mean absolute translation error of 0.297 cm and a mean absolute rotation error of 0.017 degrees on the KITTI dataset, significantly outperforming prior learning-based methods while being robust to large initial miscalibrations.

Key insights

  • Explicitly modeling feature correlation between RGB and projected depth images using a cost volume, inspired by optical flow networks, significantly improves matching accuracy for heterogeneous sensors.
  • A coarse-to-fine iterative refinement approach, utilizing multiple networks trained for decreasing miscalibration ranges, dramatically boosts the overall precision of the calibration process.
  • Predicting the deviation from an initial calibration, rather than the full extrinsic, coupled with a hybrid loss function (regression and point cloud distance), enhances robustness and learning efficiency.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.