Reproduce “Unifying Voxel-based Representation with Transformer for 3D Object Detection” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yanwei Li, Yilun Chen, Xiaojuan Qi, Zeming Li, Jian Sun, Jiaya Jia
UVTR unifies camera and LiDAR 3D object detection by mapping both modalities into a shared, height-preserving voxel space and decoding objects with a transformer. On nuScenes, it reaches 71.1% NDS and 67.1% mAP with camera–LiDAR fusion, while LiDAR-guided distillation raises camera-only performance to 55.1% NDS and 47.2% mAP on the test set.
Key insights
- Preserving vertical voxel structure is especially useful for cameras: increasing voxel height in the ablation raises camera performance from 31.4% to 35.6% NDS.
- Three-dimensional convolution is important for connecting projected image features across space; removing the voxel encoder reduces camera performance to 12.0% NDS and 2.5% mAP.
- A shared voxel coordinate system makes fusion, temporal alignment, synchronized augmentation, and cross-modal distillation components of the same representation rather than separate alignment problems.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.