Reproduce “RationalVLA: A Rational Vision-Language-Action Model with Dual System” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Wenxuan Song, Jiayi Chen, Wenxue Li, Xu He, Han Zhao, Can Cui, Pengxiang Ding Shiyan Su, Feilong Tang and 7 more
RationalVLA introduces a dual-system vision-language-action model and a new benchmark (RAMA) to enable robots to discern and reject defective natural language instructions while effectively executing valid, unseen commands. The model achieves superior performance in handling both types of instructions, demonstrating enhanced robustness and generalization in simulation and real-world manipulation tasks.
Key insights
- Explicitly training models to reject defective instructions via a dedicated rejection token is crucial for building rational and robust robotic systems.
- A dual-system architecture that separates high-level reasoning from low-level control, while allowing for effective information transfer via latent embeddings, can overcome limitations of monolithic VLAs and decoupled VLM-as-identifier systems.
- Augmenting datasets with diverse defective instructions and using chat-like expressions during MLLM fine-tuning significantly enhances the model's ability to understand and reason about language.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.