Reproduce “PixCuboid: Room Layout Estimation from Multi-view Featuremetric Alignment” on OpenResearch

About this paper

Gustav Hanning, Kalle Åström, Viktor Larsson

PixCuboid, developed at Lund University, estimates cuboid room layouts from multiple posed RGB images through a differentiable, optimization-based approach. The method achieved 87.2% 3D Intersection over Union (IoU) and 1.3° mean rotation error on ScanNet++ v2, substantially outperforming prior state-of-the-art methods by leveraging multi-view featuremetric alignment.

Key insights

• End-to-end learned dense features, specifically tailored through differentiable optimization, are crucial for robust geometric alignment and significantly outperform generic or off-the-shelf features.
• The synergy of multi-view featuremetric alignment with monocular edge and vanishing point costs is essential for robustly determining the cuboid's orientation, scale, and plane offsets.
• Leveraging readily available multi-view data with known camera poses provides richer geometric information, resolving global scale and enabling more robust 3D reconstruction compared to single-view approaches.

Summary by alphaXiv.