Reproduce “Hierarchical Control in Multi-Agent Games: LLM-based Planning and RL Execution” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jannik Hösch, Alessandro Sestini, Florian Fuchs, Amir Baghi, Joakim Bergdahl, Iolanda Leite, Konrad Tollmar, Jean-Philippe Barrette-LaPierre and 1 more
The paper uses Gemma 3 27B as a centralized controller assigning four separately trained reinforcement-learning skills to two agents. In a 2v2 King of the Hill game, LLM+RL won 46.4% of 1,000 episodes versus 51.5% for a behavior tree; the difference was nonsignificant (p = 0.103). Both beat Flat RL, and 60% of 15 participants preferred LLM+RL as human-like, in this game and study.
Key insights
- Combining an LLM's strategic reasoning with pretrained RL skills for execution provides a robust hierarchical system, addressing challenges like credit assignment and temporal abstraction in multi-agent environments.
- The LLM effectively tackles the 'option discovery problem' by inferring strategic phases and making context-dependent skill selections based on semantic interpretations of game states.
- LLM-guided agents exhibit more diverse and adaptable skill transitions (higher entropy) compared to traditional AI, which contributes to their higher perceived human-likeness by players.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.