Reproduce “Foundation Models for Autonomous Driving Perception: A Survey Through Core Capabilities” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Rajendramayavan Sathyam, Yueqi Li
A capability-based survey organizes foundation-model research for autonomous-driving perception around generalized knowledge, spatial understanding, multi-sensor robustness, and temporal understanding. It finds that current methods improve open-vocabulary recognition, 3D scene representation, sensor-failure handling, and future-state prediction, but no reviewed system yet combines all four capabilities with the latency, reliability, calibration, and verifiability required for deployment.
Key insights
- General-purpose visual and language models can provide semantic transfer, open-vocabulary recognition, explanation, and reasoning, but they require geometric grounding to support precise 3D perception and remain vulnerable to hallucination.
- Occupancy models, neural rendering, Gaussian representations, and masked 3D pretraining reduce dependence on object-level labels and describe free space, unknown regions, and scene structure more completely than isolated detections.
- Sensor fusion and temporal modeling improve robustness to missing modalities, occlusion, motion, and multiple possible futures, while calibration errors, inference cost, temporal drift, and uncertain agent behavior remain central obstacles.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.