AI Greenferencing: Routing AI Inferencing to Green Modular Data Centers with Heron

TL;DR

The paper tackles rising AI compute power demand by proposing AI Greenferencing, which routes AI inference to wind-farm co-located modular data centers. It introduces Heron, a cross-site router and a two-tier planner (Planner-L and Planner-S) that exploit wind power complementarity and workload predictability to maximize AI goodput while respecting power and latency constraints. Through profiling of 9 request classes, ILP-based planning, and real traces from Azure paired with wind-power data, the work demonstrates up to 1.8x improvements in goodput and substantial latency reductions, with scalability across many wind sites. The approach offers a practical, scalable path to green AI infrastructure by shifting compute closer to cheap renewable energy sources and intelligently routing workloads to balance variability and demand.

Abstract

AI power demand is growing unprecedentedly thanks to the high power density of AI compute and the emerging inferencing workload. On the supply side, abundant wind power is waiting for grid access in interconnection queues. In this light, this paper argues bringing AI workload to modular compute clusters co-located in wind farms. Our deployment right-sizing strategy makes it economically viable to deploy more than 6 million high-end GPUs today that could consume cheap, green power at its source. We built Heron, a cross-site software router, that could efficiently leverage the complementarity of power generation across wind farms by routing AI inferencing workload around power drops. Using 1-week ofcoding and conversation production traces from Azure and (real) variable wind power traces, we show how Heron improves aggregate goodput of AI compute by up to 80% compared to the state-of-the-art.

Figures (17)

• Figure 1: AI Greenferencing with Heron.
• Figure 2: (a) Wind capacity ($100+$ MW farms only) within a $50$ ms (emulated) fiber RTT of Azure locations as of June '$24$. (b) A significant fraction of this peak capacity lies within $20$ ms fiber RTT.
• Figure 3: Cost of power over the lifetime of an NVIDIA H$100$ GPU could be a significant fraction of its CAPEX.
• Figure 4: Comparable C/P at wind sites.
• Figure 5: Deploying less compute ($20^{th}$ ptile of generation) at the larger farms ($x$-axis) translates to less fragmentation (right-$y$) but a large number of GPUs in aggregate (left-$y$).