Reproduce “RoboWits: Unexpected Challenges for Robotic Creative Problem Solving” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chunru Lin, Hongxin Zhang, Fenghao Yu, Zhehuan Chen, Thomas L. Griffiths, Yejin Choi, David Held, Chuang Gan
RoboWits, a new bi-manual robotic benchmark, introduces a systematic framework for evaluating cognitive reasoning and creative problem-solving in unexpected manipulation scenarios. It uses an automated multi-agent pipeline to generate 208 diverse tasks, revealing that current Vision-Language-Action (VLA) models struggle with adapting to unforeseen challenges, while VLM-driven modular planners show superior robustness when operating in a closed-loop.
Key insights
- Current Vision-Language-Action (VLA) models struggle significantly with generalizing reasoning and adapting strategies when faced with unexpected environmental changes or deceptive constraints, showing a drastic performance collapse on mutated tasks.
- VLM-driven modular planners, particularly when operating in a closed-loop with real-time feedback, demonstrate superior robustness and adaptability compared to end-to-end VLAs for cognitive problem-solving tasks.
- The primary bottleneck for current robotic intelligence in complex, dynamic scenarios is poor reasoning and adaptation to novel situations, rather than solely limitations in low-level motor control.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.