Reproduce “The Change You Want to See (Now in 3D)” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ragav Sachdeva, Andrew Zisserman
A method identifies and localizes physical changes in real-world 3D scenes from two RGB images captured at different viewpoints and times, effectively accounting for occlusions and dis-occlusions. This approach demonstrates zero-shot generalization from synthetic training data, outperforming baselines on real-world 3D change detection.
Key insights
- The strategy of using a frozen, self-supervised DINO backbone and training a U-Net decoder solely on feature differences facilitates robust zero-shot generalization from synthetic to real-world data.
- Explicitly accounting for 3D geometry and occlusions/dis-occlusions through estimated 3D transformations and differentiable rendering is critical for accurate change detection across significant viewpoint changes.
- Open-set, class-agnostic physical change detection in complex 3D scenes is achievable with minimal input (two RGB images) by robustly comparing aligned visual features.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.