Reproduce “Convergence of Learning Dynamics in Stackelberg Games” on OpenResearch

About this paper

Tanner Fiez, Benjamin Chasnov, Lillian J. Ratliff

The paper establishes theoretical convergence guarantees for gradient-based learning dynamics in continuous Stackelberg games, a hierarchical game setting. It introduces new learning rules and an equilibrium concept, demonstrating that in zero-sum games (like GANs), stable learning dynamics can converge to Stackelberg equilibria, including non-Nash attractors previously considered problematic, while improving training stability.

Key insights

• In zero-sum games, the proposed Stackelberg gradient dynamics inherently converge only to differential Stackelberg equilibria, avoiding non-Nash attractors that plague simultaneous gradient play.
• Many non-Nash attracting critical points of simultaneous gradient play in zero-sum games (such as GANs) are shown to be legitimate and performant differential Stackelberg equilibria.
• The leader's update rule, incorporating second-order information from the follower's cost, effectively mitigates rotational dynamics and improves convergence stability in competitive learning settings like GANs.

Summary by alphaXiv.