Reproduce “Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Game” on OpenResearch

About this paper

Zelai Xu, Chao Yu, Fei Fang, Yu Wang, Yi Wu

Researchers from Tsinghua University and Carnegie Mellon University developed a framework combining large language models (LLMs) with reinforcement learning (RL) to create strategic agents for the complex social deduction game Werewolf. These agents effectively mitigate the intrinsic bias found in pure LLM action selection, achieving human-level performance in scenarios requiring nuanced language and deceptive communication.

Key insights

• Reinforcement learning can effectively mitigate the intrinsic bias of large language models in action selection within unbounded language action spaces, leading to less predictable and more optimal strategic play.
• Decomposing complex social deduction into structured hidden role deduction, diverse language action generation, and an RL-driven action selection process significantly enhances agent performance in deceptive communication.
• A trained RL policy, decoupled from the specific LLM used for language generation, demonstrates zero-shot transferability and generalization across different LLMs and game configurations, indicating a modular approach to strategic AI.

Summary by alphaXiv.