Reproduce “Meta-Inverse Reinforcement Learning for Mean Field Games via Probabilistic Context Variables” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yang Chen, Xiao Lin, Bo Yan, Libo Zhang, Jiamou Liu, Neset Özkan Tan, Michael Witbrock

The paper introduces Probabilistic Embeddings for Meta-Mean Field IRL (PEMMFIRL), a framework extending Mean Field Games with probabilistic context variables to infer diverse reward functions for heterogeneous agents in large-scale multi-agent systems. The method demonstrates superior policy recovery in simulations and increased taxi driver profits by 2.8% to 3.1% in a real-world spatial pricing application, effectively addressing agent heterogeneity and scalability.

Key insights

  • Introducing probabilistic context variables *externally* to the Mean Field Game (MFG) framework can effectively model agent heterogeneity while preserving the tractability and theoretical properties of MFGs.
  • Combining meta-Inverse Reinforcement Learning (meta-IRL) with MFGs provides a scalable solution for learning diverse reward functions in large populations where individual agent types are unknown.
  • An adversarial learning approach, regularized by mutual information, can jointly infer latent agent contexts and their corresponding reward functions from a mixed set of expert demonstrations.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.