Reproduce “DeWorldSG: Depth-Aware 3D Semantic Scene Graph Generation via World-Model Priors” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Seok-Young Kim, Abdelrahman Elskhawy, Taewook Ha, Dooyoung Kim, Eunjae Shin, Benjamin Busam, Woontack Woo
DeWorldSG generates spatio-temporally robust 3D Semantic Scene Graphs from RGB-D sequences by estimating instance-level 3D Gaussian distributions with depth-guided filtering and refining relations using contextual priors from a video world model. This approach improved relationship recall by 77.4% on 3DSSG and 71.3% on ReplicaSSG, demonstrating enhanced consistency and robustness in complex scenes.
Key insights
- Representing objects as probabilistic 3D Gaussians, derived from depth-refined pixel samples, provides a more stable and geometrically grounded representation than traditional 2D bounding box projections.
- Aggregating spatiotemporal evidence and using world-model priors for relation refinement significantly mitigates relational sparsity and inconsistency, leading to more contextually coherent scene graphs.
- Uncertainty-aware fusion allows judicious integration of world-model priors, complementing missing visual relations without overriding strong evidence and improving robustness to real-world noise.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.