Datacenter Cooling with Deep Reinforcement Learning

Evaporative cooling towers are responsible for billions of litres of freshwater use a year. Radiative sky cooling is a water-free alternative for heat rejection, but its capacity depends on the weather, which makes it hard to control.

A deep RL framework for adaptive control of hybrid radiative-evaporative cooling with thermal storage. DDPG, TD3 and SAC agents learn to jointly optimise radiative panel utilisation and storage dispatch from real-time weather. Evaluated on 1 MW datacenter simulations across three US climate zones — Phoenix, Houston and Seattle — using TMY3 data with 3-seed validation.

Results

• SAC reaches 94.4% ± 0.5% water savings in Seattle
• TD3 reaches 70.8% ± 4.3% in Houston
• DDPG gets the highest electricity savings in Houston, at 10.8%
• Every RL agent beats both fixed baselines and a rule-based heuristic given perfect forecasts

The ablation is the more interesting result: forecast integration is worth +10.2 percentage points in Seattle, +2.7 in Phoenix, and −1.1 in Houston. Forecasts help where the weather actually varies, and do nothing where it is consistently humid.

Known limits: this is simulation, so there is a sim-to-real gap; there is no MPC baseline yet; the ablation assumes perfect forecasts; and 48-hour episodes may be too short to learn seasonal patterns.

Next: Decision Transformers (offline RL) and Dreamer-style world models on the EnergyBench dataset (1.2 billion observations), plus carbon-aware optimisation and multi-agent coordination across datacenter clusters.

Stack: Python, Sinergym, Stable-Baselines3, DDPG/SAC/TD3, PyTorch

Related: "The Digital Dust Bowl" on Medium connects datacenter cooling to the Aral Sea crisis.