Reproduce “Robot Utility Models: General Policies for Zero-Shot Deployment in New Environments” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Haritheja Etukuru, Norihito Naka, Zijin Hu, Seungjae Lee, Julian Mehu, Aaron Edsinger, Chris Paxton, Soumith Chintala and 2 more
Robot Utility Models (RUMs) present a framework for enabling robots to perform common household tasks in new environments without additional training. The system achieved a 90% average success rate across 25 novel real-world settings by combining diverse data collection, multi-modal policy learning, and mLLM-based self-verification.
Key insights
- Data quantity and environmental diversity are more crucial than model architecture sophistication for achieving zero-shot generalization in robot policies.
- Collecting diverse data from many environments outperforms acquiring more demonstrations from fewer environments for generalization.
- Multimodal large language models (mLLMs) can effectively detect task failures and enable autonomous retries, significantly improving overall success rates in novel environments.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.