Reproduce “LEXI-SG: Monocular 3D Scene Graph Mapping with Room-Guided Feed-Forward Reconstruction” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Christina Kassab, Hyeonjae Gil, Matías Mattamala, Ayoung Kim, Maurice Fallon
LEXI-SG provides a unified monocular mapping framework that generates dense, open-vocabulary 3D scene graphs from only RGB camera input, employing a room-guided feed-forward reconstruction strategy. The system achieves the lowest average trajectory error and improved dense reconstruction quality compared to baselines, while constructing hierarchical scene representations from rooms to objects.
Key insights
- Reconstructing entire room segments in a single batch with feed-forward models leads to significantly better geometric consistency and reduced drift compared to continuously updating sliding windows.
- Global optimization of room reference poses over the Sim(3) Lie group effectively addresses monocular scale ambiguity and ensures robust alignment of local room reconstructions into a globally consistent scene.
- Combining vision-only room transition detection (using DINO features) with open-vocabulary 2D mask generation and tracking allows for comprehensive semantic understanding and 3D object lifting from purely monocular input.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.