Reproduce “CompletionFormer: Depth Completion with Convolutions and Vision Transformers” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhang Youmin, Guo Xianda, Poggi Matteo, Zhu Zheng, Huang Guan, Mattoccia Stefano
CompletionFormer presents a single-branch, hybrid convolutional and Vision Transformer architecture for dense depth map prediction from sparse measurements. This model effectively integrates local feature extraction with global context reasoning, achieving state-of-the-art performance on the KITTI Depth Completion and NYUv2 datasets while demonstrating improved computational efficiency.
Key insights
- A parallel connection between convolutional attention and Vision Transformer layers within the Joint Convolutional Attention and Transformer (JCAT) block is more effective for depth completion than cascaded designs.
- Early fusion of RGB and sparse depth into a single-branch network yields superior performance and computational efficiency compared to dual-branch architectures for multi-modal depth completion.
- Enhancing convolutional layers with spatial and channel attention within the JCAT block is crucial for effectively bridging the semantic gap between convolutional and Transformer features and preserving fine-grained local details.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.