Reproduce “FALCON: Actively Decoupled Visuomotor Policies for Loco-Manipulation with Foundation-Model-Based Coordination” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Chengyang He, Ge Sun, Yue Bai, Junkai Lu, Jiadong Zhao, Guillaume Sartoretti

FALCON enables robust loco-manipulation for legged robots by actively decoupling locomotion and manipulation into separate visuomotor policies. A frozen Vision-Language Model acts as a semantic coordinator, allowing the system to achieve 87% and 100% success rates on complex manipulation and mobile manipulation tasks respectively, outperforming centralized baselines.

Key insights

  • Actively decoupling locomotion and manipulation in legged robots, rather than using a monolithic policy, improves robustness and generalization for complex, asymmetric tasks.
  • Foundation models can be effectively leveraged as lightweight semantic coordinators, utilizing their pre-trained knowledge to guide sub-policies without requiring extensive end-to-end training.
  • Language-defined phase and progress signals, combined with contrastive learning, offer effective mechanisms to ensure temporal coherence and mutual consistency within decoupled robotic architectures.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.