Reproduce “2D-3D Geometric Fusion Network using Multi-Neighbourhood Graph Convolution for RGB-D Indoor Scene Classification” on OpenResearch

About this paper

Albert Mosella-Montoro, Javier Ruiz-Hidalgo

A 2D-3D Geometric Fusion Network improves RGB-D indoor scene classification by integrating novel Multi-Neighbourhood Graph Convolution (MUNEGC) and Nearest Voxel Pooling (NVP) for robust 3D geometric feature extraction with 2D texture features. The approach achieves a mean accuracy of 58.6% on SUN RGB-D and 75.1% on NYU-Depth-V2, surpassing previous state-of-the-art results by 1.9% and 5.9% respectively.

Key insights

• Leveraging both spatial (Euclidean) and feature similarity (Feature) neighborhoods within a graph convolution operation significantly enhances the robustness of 3D geometric feature extraction.
• Accurate 3D downsampling is achieved by grouping points based on their true geometric proximity (NVP) rather than fixed voxel boundaries, mitigating noise and improving feature fidelity.
• A geometrically-aware fusion strategy, which spatially aligns 2D texture features with 3D geometric features in the 3D domain, is more effective than simpler late-fusion approaches.

Summary by alphaXiv.