Reproduce “SceneGlue: Scene-Aware Transformer for Feature Matching without Scene-Level Annotation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Songlin Du, Xiaoyong Lu, Yaping Yan, Guobao Xiao, Xiaobo Lu, Takeshi Ikenaga
SceneGlue introduces a Transformer-based framework for feature matching that integrates multi-scale feature representation, parallel attention, and a Visibility Transformer to explicitly estimate cross-view visible regions. This architecture yields improved accuracy and robustness across various computer vision tasks, including homography and pose estimation, without requiring scene-level annotations.
Key insights
- Dynamically encoding position with a wave model (Wave-PE) provides a more effective and robust way to infuse spatial context into descriptors than static position encoding methods.
- Computing self- and cross-attention in parallel, coupled with attention weight sharing, improves the efficiency and effectiveness of feature interaction, leading to richer implicit scene-aware representations.
- Explicitly modeling cross-view visible regions via a classification task, supervised only by local matches, offers interpretable scene-level awareness that guides the model to focus on relevant areas, significantly enhancing matching performance.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.