Reproduce “RoboHiMan: A Hierarchical Evaluation Paradigm for Compositional Generalization in Long-Horizon Manipulation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yangtao Chen, Zixuan Chen, Nga Teng Chan, Junting Chen, Junhui Yin, Jieqi Shi, Yang Gao, Yong-Lu Li and 1 more
RoboHiMan introduces a hierarchical evaluation framework and benchmark (HiMan-Bench) for long-horizon robotic manipulation, assessing compositional generalization under diverse environmental perturbations. The research identifies that current visual-language-action (VLA) models struggle with composing skills and robustly generalizing to unseen tasks, emphasizing the need for explicit planning and addressing error accumulation in hierarchical systems.
Key insights
- Explicit planning is essential for robust skill composition, as non-hierarchical models achieved near-zero success on complex compositional tasks.
- Scaling training data for atomic and compositional skills only partially addresses generalization, as models still struggle with unseen compositional tasks.
- Hierarchical systems face significant performance degradation due to error accumulation from both high-level planner inaccuracies and low-level policy execution failures, necessitating robust feedback mechanisms.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.