Reproduce “Controlling Commercial Cooling Systems Using Reinforcement Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Jerry Luo, Cosmin Paduraru, Octavian Voicu, Yuri Chervonyi, Scott Munns, Jerry Li, Crystal Qian, Praneet Dutta and 28 more

Researchers from DeepMind, Google, and Trane developed BCOOLER, a reinforcement learning system to optimize commercial cooling systems. Deployed in two real facilities, the system achieved energy savings of 9% and 13% against heuristic controllers while maintaining occupant comfort and system safety.

Key insights

  • Reinforcement learning can effectively optimize complex, multi-constrained, real-world industrial control problems like chiller plant operations without requiring explicit physics-based models.
  • The use of ensemble models for uncertainty estimation allows for conservative constraint satisfaction (pessimistic estimates for safety) and targeted exploration (optimistic estimates in uncertain areas) in safety-critical applications.
  • Continuous daily retraining and robust real-world mitigations (e.g., sensitivity analysis, feature engineering, hierarchical policies, constraint tweaking) are essential to address challenges like limited data, non-stationary dynamics, and complex constraints in live deployments.
  • RL agents can implicitly learn to compensate for real-world system nuances, such as sensor miscalibration, by observing outcomes and adjusting setpoints accordingly.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.