Reproduce “All You Need for Object Detection: From Pixels, Points, and Prompts to Next-Gen Fusion and Multimodal LLMs/VLMs in Autonomous Vehicles” on OpenResearch

About this paper

Sayed Pedram Haeri Boroujeni, Niloufar Mehrabi, Hazim Alzorgan, Mahlagha Fazeli, Abolfazl Razi

This survey offers a comprehensive overview of object detection for autonomous vehicles, systematically analyzing AV sensors, simulators, a novel taxonomy of datasets, and detection methodologies. The work at Clemson University emphasizes multimodal fusion strategies and the integration of emerging Vision-Language Models (VLMs) and Large Language Models (LLMs) to identify current capabilities and future research opportunities.

Key insights

• Robust object detection in autonomous vehicles fundamentally relies on multimodal sensor fusion to overcome individual sensor limitations, with mid-level fusion presenting a strong balance between information richness and computational efficiency.
• Autonomous vehicle simulators are critical for generating diverse data, creating challenging scenarios, and conducting reproducible evaluations, especially for rare and unsafe conditions.
• Emerging Vision-Language Models (VLMs) and Large Language Models (LLMs) offer capabilities for semantic understanding, open-vocabulary detection, and contextual reasoning beyond traditional methods, enhancing interpretability and generalization.

Summary by alphaXiv.