Reproduce “MergeVLA: Cross-Skill Model Merging Toward a Generalist Vision-Language-Action Agent” on OpenResearch

About this paper

Yuxia Fu, Zhizhen Zhang, Yuqi Zhang, Zijian Wang, Zi Huang, Yadan Luo

MergeVLA is a framework designed to overcome the challenge of merging independently fine-tuned Vision-Language-Action (VLA) models, which previously failed due to structural specializations. The framework achieves robust multi-skill, multi-embodiment, and multi-environment generalization, demonstrating up to a 90.2% average success rate on the LIBERO benchmark and successful application on a real robotic arm.

Key insights

• VLA model non-mergeability is primarily caused by divergent LoRA updates in the VLM backbone and architectural coupling in action experts, where task-specific information propagates widely across layers.
• Sparsely activating LoRA parameters through task-specific masking can effectively preserve pretrained visual-language representations and mitigate cross-task interference during merging.
• Removing self-attention from the action expert forces reliance on robust VLM features, making shallow layers more amenable to merging, while deep layers (expert heads) remain task-specialized and should be retained individually.
• Value projection matrices from the action expert provide stable and discriminative signals for test-time task routing, enabling dynamic selection of skill components based on input features without additional training.

Summary by alphaXiv.