Reproduce “ASALT: Adaptive State Alignment for Lateral Transfer in Multi-agent Reinforcement Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Anurag Akula, Satheesh K. Perepu, Abhishek Sarkar, Kaushik Dey

The ASALT framework explicitly accommodates mismatched observation and global state space dimensionalities in multi-agent reinforcement learning transfer learning. It enables faster convergence and robust performance in diverse multi-agent environments by employing adaptive state alignment and lateral knowledge transfer, including actor and critic information.

Key insights

  • Jointly training state and observation adapters with the target agent substantially accelerates convergence compared to independent training.
  • Transferring knowledge from both actor and critic networks, facilitated by state adapters, provides more accurate advantage estimates and generally improves learning performance.
  • Attention-based observation adapters effectively capture and transfer complex inter-agent coordination patterns, leading to better performance in varied scenarios, especially up-scaling.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.