Reproduce “Robot Fleet Learning via Policy Merging” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Lirui Wang, Kaiqing Zhang, Allan Zhou, Max Simchowitz, Russ Tedrake
Researchers from MIT CSAIL and University of Maryland introduced FLEET-MERGE, an algorithm for merging locally trained recurrent neural network (RNN) policies from robot fleets, addressing permutation invariances to enable efficient skill consolidation without centralizing raw data. Evaluated on a new FLEET-TOOLS benchmark and Meta-World, the approach achieved superior multi-task performance over baselines and comparable results to joint training while significantly reducing communication costs.
Key insights
- Successfully merging RNN-parameterized policies for robot control requires explicitly accounting for their unique permutation invariances, which FLEET-MERGE achieves through iterative soft/hard projection of permutation matrices.
- Policy merging provides a communication-efficient "bottom-up" alternative to data centralization, enabling skill consolidation from diverse local robot experiences without sharing raw data.
- The specific group structure of policy invariances (e.g., permutations for non-linear RNNs versus general invertible transformations for linear policies) is critical for designing effective model merging algorithms.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.