Reproduce “Trajectory-Class-Aware Multi-Agent Reinforcement Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Hyungho Na, Kwanghyeon Lee, Sumin Lee, Il-Chul Moon

This work introduces a multi-agent reinforcement learning framework that enables agents to autonomously identify the type of task they are performing from partial observations in unsupervised multi-task environments. The method allows agents to learn and execute distinct, trajectory-class-dependent policies, resulting in enhanced performance and generalization across various StarCraft II multi-task benchmarks.

Key insights

  • Maintaining consistent trajectory class labels through centroid initialization during clustering is crucial for stable training of the class predictor and for agents to learn effective task-dependent behaviors.
  • The modified VQ-VAE's task-aware coverage loss effectively distributes quantized vectors across different task-specific state distributions, enhancing the model's ability to capture and differentiate key trajectory characteristics.
  • Learning a dedicated, higher-level representation for the predicted trajectory class provides a more informative signal for policy conditioning than a direct one-hot encoding, leading to improved performance.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.