Reproduce “BEV-TSR: Text-Scene Retrieval in BEV Space for Autonomous Driving” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tao Tang, Dafeng Wei, Zhengyu Jia, Tian Gao, Changwei Cai, Chengkai Hou, Peng Jia, Kun Zhan and 7 more
Researchers from Li Auto Inc. and Sun Yat-sen University developed BEV-TSR, a text-scene retrieval framework that operates in Bird's Eye View (BEV) space for autonomous driving. This framework achieves over 85% Top-1 accuracy for text-to-scene retrieval on a new multi-level `nuScenes-Retrieval` dataset by integrating BEV features, knowledge graphs, and cross-modal alignment.
Key insights
- Operating text-scene retrieval in BEV space provides a globally consistent and intuitive representation of autonomous driving scenes, significantly improving understanding and matching compared to perspective views.
- Enriching language embeddings with external knowledge graphs (KGP) effectively expands semantic representation, enabling better comprehension of complex, relational textual descriptions in AD scenarios.
- Combining contrastive learning (SCE) with an auxiliary generative task (CG) creates a robust mechanism for aligning disparate BEV visual features and complex language embeddings.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.