Reproduce “STELLAR: Scaling 3D Perception Large Models for Autonomous Driving” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yingwei Li, Xin Huang, Yang Liu, Yang Fu, Alex Zihao Zhu, Chen Song, Junwen Yao, Anant Subramanian and 8 more
Researchers at Waymo developed STELLAR, a multi-modal Transformer model designed for 3D perception in autonomous driving, which systematically analyzes the impact of scaling model parameters and data volume. The model established a new state-of-the-art on the Waymo Open Dataset, showing improved Average Precision and Average Precision with Heading metrics, particularly for cyclist detection.
Key insights
- Scaling model parameters and data volume consistently improves 3D perception performance, though exhibiting diminishing returns rather than strong log-linear scaling observed in some other large models.
- For a fixed computational budget, achieving optimal performance often requires utilizing the largest possible dataset, even if it means employing a comparatively smaller model.
- A multi-stage training strategy, combining large-scale auto-labeled pre-training, human-annotated multi-task mid-training, and specific dataset fine-tuning, significantly boosts 3D detection performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.