Reproduce “Availability-aware Sensor Fusion via Unified Canonical Space” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Dong-Hee Paek, Seung-Hyun Kong
Researchers at KAIST developed Availability-aware Sensor Fusion (ASF), a perception system for autonomous driving that projects heterogeneous sensor data into a unified canonical space and uses efficient cross-attention to dynamically adapt to sensor availability. This system achieves significant improvements in 3D object detection, particularly in adverse weather, while maintaining real-time inference speeds and robustness to sensor degradation.
Key insights
- Projecting features from heterogeneous sensor modalities into a unified canonical space effectively addresses representation inconsistency, enabling more coherent fusion.
- A computationally efficient cross-attention mechanism applied across sensors for individual spatial patches allows dynamic adaptation to sensor availability without prohibitive overhead.
- Training with a sensor combination loss explicitly prepares the model to gracefully handle scenarios of sensor degradation or failure, enhancing overall robustness in real-world conditions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.