Reproduce “LiDAR Prompted Spatio-Temporal Multi-View Stereo for Autonomous Driving” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Qihao Sun, Jiarun Liu, Ziqian Ni, Jianyun Xu, Tao Xie, Lijun Zhao, Ruifeng Li, Sheng Yang
DriveMVS, a multi-view stereo framework from CaiNiao Inc. and Alibaba Group, integrates sparse LiDAR prompts with spatio-temporal reasoning to deliver accurate, temporally consistent, and robust metric depth estimation for autonomous driving. It achieved state-of-the-art results, including a 0.49m Mean Absolute Error on KITTI and a 0.296% Temporal Alignment Error, demonstrating superior performance across diverse driving conditions.
Key insights
- Disentangling relative geometric consistency from absolute metric anchoring within the cost volume significantly improves accuracy and robustness in challenging scenes by mitigating scale ambiguity.
- The deep fusion of diverse cues (MVS geometry, monocular context, sparse LiDAR metric) using a Transformer, with attention mechanisms robust to sparsity, effectively resolves ambiguities and enhances overall depth quality.
- Explicitly modeling temporal context through motion-aware self-attention and geometric embeddings in the decoder is critical for achieving highly temporally consistent and flicker-free depth estimations in dynamic autonomous driving scenarios.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.