Reproduce “MARLIN: Multi-Agent Game-Theoretic Reinforcement Learning for Sustainable LLM Inference in Cloud Datacenters” on OpenResearch

About this paper

H. Moore, S. Qi, D. Milojicic, C. Bash, S. Pasricha

Researchers from Colorado State University and Hewlett Packard Labs developed MARLIN, a multi-agent game-theoretic reinforcement learning framework that optimizes LLM inference scheduling in geo-distributed cloud datacenters. This framework simultaneously minimizes time-to-first-token, carbon emissions, water usage, and energy costs, achieving up to 33.61% lower carbon emissions and 43.88% lower water usage compared to state-of-the-art baselines.

Key insights

• A multi-agent game-theoretic reinforcement learning approach effectively balances and co-optimizes competing performance and sustainability objectives in dynamic cloud environments.
• The framework's competitive blending process in Phase 2, supported by individual agent proposals, is critical for generating high-quality, diverse, and Pareto-optimal solutions across multiple metrics.
• Leveraging geo-distributed datacenters, with their temporal and spatial variations in carbon intensity, water intensity, and energy costs, offers substantial opportunities for improving overall LLM inference sustainability.

Summary by alphaXiv.