Reproduce “λ: A Benchmark for Data-Efficiency in Long-Horizon Indoor Mobile Manipulation Robotics” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ahmed Jaafar, Shreyas Sundara Raman, Sudarshan Harithas, Yichen Wei, Sofia Juliani, Anneke Wernerfelt, Benedict Quartey, Ifrah Idrees and 2 more
The LAMBDA benchmark was developed to evaluate the data efficiency of models performing language-conditioned, long-horizon indoor mobile manipulation (MoMa) tasks, utilizing both simulated and real-world quadruped robot data. It reveals the significant data inefficiency of current learning-based approaches and highlights the superior zero-shot performance of neuro-symbolic systems.
Key insights
- Learning-based Behavior Cloning (BC) models, such as RT-1 and MG-MoMa, exhibit substantial data inefficiency for long-horizon mobile manipulation, struggling to generalize effectively from limited data to unseen scenes or tasks.
- Neuro-symbolic approaches, exemplified by the LIMP system, demonstrate significantly higher data efficiency in a zero-shot setting by integrating semantic mapping, robust motion planning, and multimodal foundation models.
- Architectural choices can influence data efficiency, with a Mamba-based architecture (RM-1) showing consistent performance advantages over the transformer-based RT-1 across varying dataset sizes within the benchmark.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.