Reproduce “Low-Rank Agent-Specific Adaptation (LoRASA) for Multi-Agent Policy Learning” on OpenResearch

About this paper

Beining Zhang, Aditya Kapoor, Mingfei Sun

Low-Rank Agent-Specific Adaptation (LoRASA) introduces a two-phase training method for multi-agent reinforcement learning, combining a shared policy backbone with agent-specific low-rank adaptations. This approach enables specialized behaviors while achieving comparable or superior performance to existing methods, using substantially fewer parameters and reducing computational time compared to fully distinct policies.

Key insights

• Agent-specific policy deviations in cooperative MARL tasks often reside within a lower-dimensional affine subspace, allowing efficient capture through low-rank adaptations.
• Framing multi-agent policy learning as a multi-task learning problem, where each agent's policy is a distinct task, enables efficient specialization from a common knowledge base.
• Moderate LoRA ranks (e.g., r=8) balance expressivity and efficiency, often performing as well as or better than full-rank updates, while distributing adapters across multiple network layers is crucial for robust specialization.

Summary by alphaXiv.