Reproduce “Optimizing Industrial HVAC Systems with Hierarchical Reinforcement Learning” on OpenResearch

About this paper

William Wong, Praneet Dutta, Octavian Voicu, Yuri Chervonyi, Cosmin Paduraru, Jerry Luo

Researchers from Carnegie Mellon University and DeepMind introduce a hierarchical reinforcement learning (HRL) architecture to optimize industrial HVAC chiller plants, addressing issues of energy consumption and equipment wear. The HRL system achieves approximately 10% energy savings compared to flat reinforcement learning and 17% over heuristic policies while learning safe, sustainable chiller cycling behaviors.

Key insights

• Hierarchical decomposition effectively disentangles control actions across different timescales, enabling the HRL agent to learn real-world feasible behaviors (e.g., infrequent chiller cycling) that flat RL agents struggle with.
• The HRL approach reduces the need for complex and extensive reward engineering to instill human preferences and operational constraints, as it naturally promotes desirable long-term behaviors.
• HRL provides a mechanism to balance competing objectives—energy minimization, thermal comfort, and equipment longevity—more effectively than single-agent or simple multi-agent RL approaches.

Summary by alphaXiv.