Reproduce “Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xiaokang Chen, Kwan-Yee Lin, Jingbo Wang, Wayne Wu, Chen Qian, Hongsheng Li, Gang Zeng
Researchers from Peking University, SenseTime Research, and The Chinese University of Hong Kong developed a bi-directional cross-modality feature propagation system with a Separation-and-Aggregation Gate (SA-Gate) for RGB-D semantic segmentation, yielding state-of-the-art results on datasets like CityScapes (82.8% mIoU) and NYU Depth V2 (52.4% mIoU) while demonstrating enhanced robustness to noisy depth data.
Key insights
- Explicitly filtering noisy depth signals through channel-wise attention and recalibrating features using cross-modality global descriptors significantly refines individual modality representations.
- A spatially adaptive gating mechanism effectively weighs and combines RGB and depth features based on their local informational quality, addressing scene variability.
- Bi-directional multi-step propagation ensures continuous interaction between modalities across encoder layers, preserving specificity while leveraging fused information.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.