Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Green-LLM: Optimal Workload Allocation for Environmentally-Aware Distributed Inference

Jiaming Cheng, Duong Tung Nguyen

Abstract

This letter investigates the optimal allocation of large language model (LLM) inference workloads across heterogeneous edge data centers (DCs) over time. Each DC features on-site renewable generation and faces dynamic electricity prices and spatiotemporal variability in renewable availability. The central question is: how can inference workloads be optimally distributed to the DCs to minimize energy consumption, carbon emissions, and water usage while enhancing user experience? This letter proposes a novel optimization model for LLM service providers to reduce operational costs and environmental impacts. Numerical results validate the efficacy of the proposed approach.

Green-LLM: Optimal Workload Allocation for Environmentally-Aware Distributed Inference

Abstract

This letter investigates the optimal allocation of large language model (LLM) inference workloads across heterogeneous edge data centers (DCs) over time. Each DC features on-site renewable generation and faces dynamic electricity prices and spatiotemporal variability in renewable availability. The central question is: how can inference workloads be optimally distributed to the DCs to minimize energy consumption, carbon emissions, and water usage while enhancing user experience? This letter proposes a novel optimization model for LLM service providers to reduce operational costs and environmental impacts. Numerical results validate the efficacy of the proposed approach.

Paper Structure

This paper contains 11 sections, 8 equations, 2 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. System Model and Problem Formulation
  3. System Model
  4. Problem formulation
  5. Numerical Results
  6. Simulation setup
  7. Network topology
  8. LLM inference workload
  9. Other parameters
  10. Performance evaluation
  11. Conclusion

Figures (2)

  • Figure 1: Varying carbon intensity $\theta$ and renewable energy $P_w$: effective cost and carbon reduction requires joint, time-aware multi-objective optimization.
  • Figure 2: Varying token size $\mathbf{\tau}$ and delay penalty $\rho$: M0 is robust in managing inference workloads where large token sizes and delay penalties coexist.