Reproduce “BEVDistill: Cross-Modal BEV Distillation for Multi-View 3D Object Detection” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Zehui Chen, Zhenyu Li, Shiquan Zhang, Liangji Fang, Qinhong Jiang, Feng Zhao

Researchers from USTC, HIT, and SenseTime Research developed BEVDistill, a cross-modal Bird-Eye-View (BEV) distillation framework that transfers spatial knowledge from LiDAR to camera-based 3D object detectors. The method boosts camera-only 3D detection performance by +3.4 NDS on nuScenes while incurring no additional inference cost, achieving a new state-of-the-art of 59.4 NDS for published image-based detectors.

Key insights

  • Unifying features from different modalities (camera and LiDAR) into a shared Bird-Eye-View (BEV) space effectively bridges view discrepancy and enables robust cross-modal knowledge transfer for 3D object detection.
  • Traditional knowledge distillation methods, such as direct L2 feature imitation or KL divergence on classification logits, are often suboptimal or detrimental in cross-modal settings due to fundamental modality differences and can be improved with adaptive strategies.
  • Transferring richer, structured knowledge through mutual information maximization on internal representations, rather than direct logit matching, combined with foreground-guided dense distillation and quality-score re-weighting for sparse instances, is crucial for effective cross-modal learning.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.