Reproduce “Object as Query: Lifting any 2D Object Detector to 3D Detection” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zitian Wang, Zehao Huang, Jiahui Fu, Naiyan Wang, Si Liu
The MV2D framework, developed by researchers from Beihang University and TuSimple, proposes a method to adapt any 2D object detector for multi-view 3D object detection by generating dynamic 3D object queries directly from 2D detections. It achieves state-of-the-art performance on the nuScenes dataset, reaching 48.3% mAP and 57.3% NDS with a ResNet-101 backbone.
Key insights
- Leveraging 2D object detector outputs as dynamic 3D queries provides superior object location hypotheses compared to fixed, learned queries, enhancing recall and precision.
- Restricting cross-attention to only object-specific "relevant features" across views improves 3D localization accuracy by focusing information and reducing interference.
- The modular design allows MV2D to integrate any off-the-shelf 2D detector, enabling the 3D detection framework to benefit directly from continuous advancements in 2D perception.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.