Reproduce “Semantic Scene Difference Detection in Daily Life Patroling by Mobile Robots using Pre-Trained Large-Scale Vision-Language Model” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yoshiki Obinata, Kento Kawaharazuka, Naoaki Kanazawa, Naoya Yamaguchi, Naoto Tsukamoto, Iori Yanokura, Shingo Kitagawa, Koki Shinjo and 2 more
Researchers from The University of Tokyo developed a method for mobile robots to detect semantic scene differences in daily life environments by leveraging pre-trained large-scale vision-language models. This approach quantifies semantic changes between scenes without requiring local training, demonstrating sensitivity to meaningful alterations while being robust to minor visual noise.
Key insights
- Pre-trained large-scale Vision-Language Models can effectively enable semantic scene understanding and difference detection in robots without requiring local training or fine-tuning for specific environments.
- Quantifying semantic differences via natural language descriptions allows robots to distinguish meaningful environmental changes from minor, non-semantic visual variations.
- A question weighting mechanism based on semantic diversity improves the consistency and stability of scene distance measurements, reducing bias from semantically redundant questions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.