Reproduce “Safe and Interpretable Multimodal Path Planning for Multi-Agent Cooperation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Haojun Shi, Suyu Ye, Katherine M. Guerrerio, Jianzhi Shen, Yifan Yin, Daniel Khashabi, Chien-Ming Huang, Tianmin Shu

Researchers from Johns Hopkins University developed CaPE (Code as Path Editor), a framework that enables robots to safely and interpretably adjust their path plans in multi-agent cooperation through natural language instructions. The system combines vision-language models for synthesizing path editing programs with a model-based planner and a verifier, achieving up to 90% success in multi-robot coordination and 70% in real-world human-robot joint carrying tasks.

Key insights

  • Translating natural language into explicit, verifiable path editing programs in a DSL provides both safety guarantees through verification and enhanced interpretability of robot actions.
  • Generating multiple, qualitatively distinct candidate paths using homotopy-aware planning significantly improves the VLM's ability to select and modify appropriate routes based on human instructions.
  • Fine-tuning smaller VLMs on synthetic data for specific tasks like path editing program synthesis can achieve high performance and efficiency, making such systems practical for real-world robotic deployment.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.