Reproduce “Knowing What Not to Do: Leverage Language Model Insights for Action Space Pruning in Multi-agent Reinforcement Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhihao Liu, Xianliang Yang, Zichuan Liu, Yifan Xia, Wei Jiang, Yuanyu Zhang, Lijuan Li, Guoliang Fan and 2 more
The eSpark framework leverages Large Language Models to generate and refine exploration functions for intelligent action space pruning in Multi-agent Reinforcement Learning. This approach enhanced the performance of base MARL algorithms by an average of 34.4% in multi-agent inventory management and 9.9% in traffic signal control across diverse industrial scenarios.
Key insights
- Large Language Models possess surprising semantic understanding, reasoning, and planning abilities that can be leveraged to generate effective exploration functions for action space pruning in MARL.
- Intelligently pruning the action space during training can significantly enhance MARL efficiency, leading to faster convergence and better overall performance by guiding exploration towards more relevant actions.
- An iterative framework combining LLM-driven knowledge generation, evolutionary search, and policy feedback, along with retention training, allows for continuous refinement of exploration strategies, proving more adaptive than fixed heuristics.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.