Reproduce “GenNBV: Generalizable Next-Best-View Policy for Active 3D Reconstruction” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Xiao Chen, Quanyi Li, Tai Wang, Tianfan Xue, Jiangmiao Pang

GenNBV introduces a generalizable Next-Best-View (NBV) policy for active 3D reconstruction, utilizing a 5D free-space action domain and a multi-source state embedding derived directly from raw sensory inputs. The method demonstrates robust cross-dataset generalization, achieving 91.19% Mean AUC and 98.26% Final Coverage Ratio on unseen building-scale geometries from the Houses3K test set, and successfully generalizing to diverse non-house categories.

Key insights

  • Operating in a 5D free-space action domain significantly enhances the policy's ability to explore complex geometries and capture optimal viewpoints, leading to more complete reconstructions than methods with constrained action spaces.
  • A multi-source state embedding, particularly the probabilistic 3D grid that explicitly models 'unknown' space, is crucial for guiding intelligent exploration and achieving robust cross-dataset generalization without per-scene optimization.
  • Reinforcement Learning, combined with diverse training environments and efficient simulation (e.g., Isaac Gym), can yield highly generalizable policies capable of adapting to unseen objects and even different categories, advancing autonomous 3D reconstruction.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.