Reproduce “Clutter-Robust Vision-Language-Action Models through Object-Centric and Geometry Grounding” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Khoa Vo, Taisei Hanyu, Yuki Ikebe, Trong Thang Pham, Nhat Chung, Minh Nhat Vu, Duy Nguyen Ho Minh, Anh Nguyen and 3 more
Researchers from the University of Arkansas and collaborators developed OBEYED-VLA, a framework that enhances Vision-Language-Action (VLA) models by decoupling perceptual grounding from action reasoning using object-centric and geometry-aware observations. This approach allows VLA policies, fine-tuned only on single-object demonstrations, to achieve up to 90% success in highly cluttered environments with multiple distractors and effectively generalize to unseen objects, outperforming baselines by substantial margins while rejecting absent targets with 95% accuracy.
Key insights
- Explicitly decoupling perceptual grounding from action reasoning allows VLA policies to effectively handle visual clutter and distractors, as the perception module filters out irrelevant information before action prediction.
- Combining object-centric grounding (identifying task-relevant objects) with geometry-aware grounding (using depth information) provides robust, appearance-invariant representations that improve generalization to unseen objects and diverse backgrounds.
- Robustness in cluttered scenes and generalization to novel conditions can be achieved by fine-tuning VLA policies exclusively on simple, single-object demonstrations, provided a strong, modular perception grounding module is in place.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.