DCoPilot: Generative AI-Empowered Policy Adaptation for Dynamic Data Center Operations

TL;DR

DCoPilot addresses the specification-to-policy latency in dynamic data center cooling and IT control by integrating an LLM-driven symbolic reward generator with a hypernetwork-based parametric policy generator. The offline pipeline uses simulation-scale reward evolution and trajectory-led distillation to train a hypernetwork that can produce deployment-ready policy weights for new specifications in zero-shot fashion, enabling minute-scale control with near-zero SLA violations. Across five DC task families, it outperforms baselines, showing robust generalization and stability during specification changes, and ablation confirms the importance of unified reward generation and boundary-guided optimization. The approach promises safer, more energy-efficient DC operation by dramatically reducing adaptation latency and improving resilience to evolving workloads and SLAs.

Abstract

Modern data centers (DCs) hosting artificial intelligence (AI)-dedicated devices operate at high power densities with rapidly varying workloads, making minute-level adaptation essential for safe and energy-efficient operation. However, manually designing piecewise deep reinforcement learning (DRL) agents cannot keep pace with frequent dynamics shifts and service-level agreement (SLA) changes of an evolving DC. This specification-to-policy lag causes a lack of timely, effective control policies, which may lead to service outages. To bridge the gap, we present DCoPilot, a hybrid framework for generative control policies in dynamic DC operation. DCoPilot synergizes two distinct generative paradigms, i.e., a large language model (LLM) that performs symbolic generation of structured reward forms, and a hypernetwork that conducts parametric generation of policy weights. DCoPilot operates through three coordinated phases: (i) simulation scale-up, which stress-tests reward candidates across diverse simulation-ready (SimReady) scenes; (ii) meta policy distillation, where a hypernetwork is trained to output policy weights conditioned on SLA and scene embeddings; and (iii) online adaptation, enabling zero-shot policy generation in response to updated specifications. Evaluated across five control task families spanning diverse DC components, DCoPilot achieves near-zero constraint violations and outperforms all baselines across specification variations. Ablation studies validate the effectiveness of LLM-based unified reward generation in enabling stable hypernetwork convergence.

Figures (11)

• Figure 1: DCoPilot replaces manual, piecewise DRL-based policies with a generative pipeline: an LLM formalizes operational specifications into a symbolic family reward, and a hypernetwork generates parametric policy network weights under evolving environments.
• Figure 2: Adaptation performance of continuous learning approaches at day-level and minute-level evaluations.
• Figure 3: General topology of DC infrastructure, including server rooms, chiller plant, electrical system, and power grid. DC operators continuously install new servers and adjust SLAs, making homogeneous changes over time.
• Figure 4: The hierarchical structure of a DC SimReady scene. A DC SimReady scene consists of three-level SimReady assets. Each SimReady asset is a surrogate model trained from operational data.
• Figure 5: Given scene $G$ and operator objective $I_{\text{ops}}$, the Reward LLM generates reward code candidates (➊). Each candidate is evaluated on boundary conditions by training policies and collecting trajectories (➋-➌). Then, it ranks candidates based on SLA violation costs and objective scores, selecting top-$k$ for evolutionary refinement (➍). The optimal reward form $\bar{R}^*$ trains DRL agents across sampled specifications $\mathbf{e} = \bm{\mu} \otimes \bm{\psi}$ (➎), generating near-optimal trajectory pool $\mathbf{T}$ for hypernetwork distillation (➏).