Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Online GPU Energy Optimization with Switching-Aware Bandits

Xiongxiao Xu, Solomon Abera Bekele, Brice Videau, Kai Shu

TL;DR

EnergyUCB is proposed, a lightweight UCB-based controller that dynamically adjusts GPU core frequency in real time to save energy and incorporates a switching-aware UCB index and a QoS-constrained variant that enforce explicit slowdown budgets while discouraging unnecessary frequency oscillations.

Abstract

Energy consumption has become a bottleneck for future computing architectures, from wearable devices to leadership-class supercomputers. Existing energy management techniques largely target CPUs, even though GPUs now dominate power draw in heterogeneous high performance computing (HPC) systems. Moreover, many prior methods rely on either purely offline or hybrid offline and online training, which is impractical and results in energy inefficiencies during data collection. In this paper, we introduce a practical online GPU energy optimization problem in a HPC scenarios. The problem is challenging because (1) GPU frequency scaling exhibits performance-energy trade-offs, (2) online control must balance exploration and exploitation, and (3) frequent frequency switching incurs non-trivial overhead and degrades quality of service (QoS). To address the challenges, we formulate online GPU energy optimization as a multi-armed bandit problem and propose EnergyUCB, a lightweight UCB-based controller that dynamically adjusts GPU core frequency in real time to save energy. Specifically, EnergyUCB (1) defines a reward that jointly captures energy and performance using a core-to-uncore utilization ratio as a proxy for GPU throughput, (2) employs optimistic initialization and UCB-style confidence bonuses to accelerate learning from scratch, and (3) incorporates a switching-aware UCB index and a QoS-constrained variant that enforce explicit slowdown budgets while discouraging unnecessary frequency oscillations. Extensive experiments on real-world workloads from the world's third fastest supercomputer Aurora show that EnergyUCB achieves substantial energy savings with modest slowdown and that the QoS-constrained variant reliably respects user-specified performance budgets.

Online GPU Energy Optimization with Switching-Aware Bandits

TL;DR

EnergyUCB is proposed, a lightweight UCB-based controller that dynamically adjusts GPU core frequency in real time to save energy and incorporates a switching-aware UCB index and a QoS-constrained variant that enforce explicit slowdown budgets while discouraging unnecessary frequency oscillations.

Abstract

Energy consumption has become a bottleneck for future computing architectures, from wearable devices to leadership-class supercomputers. Existing energy management techniques largely target CPUs, even though GPUs now dominate power draw in heterogeneous high performance computing (HPC) systems. Moreover, many prior methods rely on either purely offline or hybrid offline and online training, which is impractical and results in energy inefficiencies during data collection. In this paper, we introduce a practical online GPU energy optimization problem in a HPC scenarios. The problem is challenging because (1) GPU frequency scaling exhibits performance-energy trade-offs, (2) online control must balance exploration and exploitation, and (3) frequent frequency switching incurs non-trivial overhead and degrades quality of service (QoS). To address the challenges, we formulate online GPU energy optimization as a multi-armed bandit problem and propose EnergyUCB, a lightweight UCB-based controller that dynamically adjusts GPU core frequency in real time to save energy. Specifically, EnergyUCB (1) defines a reward that jointly captures energy and performance using a core-to-uncore utilization ratio as a proxy for GPU throughput, (2) employs optimistic initialization and UCB-style confidence bonuses to accelerate learning from scratch, and (3) incorporates a switching-aware UCB index and a QoS-constrained variant that enforce explicit slowdown budgets while discouraging unnecessary frequency oscillations. Extensive experiments on real-world workloads from the world's third fastest supercomputer Aurora show that EnergyUCB achieves substantial energy savings with modest slowdown and that the QoS-constrained variant reliably respects user-specified performance budgets.

Paper Structure

This paper contains 20 sections, 8 equations, 5 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. The Aurora Node Architecture
  4. Multi-Armed Bandits
  5. Problem Definition
  6. Methodology
  7. Modeling GPU Online Energy Consumption
  8. EnergyUCB Algorithm
  9. Constrained EnergyUCB for QoS Guarantee
  10. Experiments
  11. Experimental Setup
  12. Experimental Results
  13. Ablation Study
  14. Switching Cost Analysis
  15. Reward Formulation Analysis
  16. ...and 5 more sections

Figures (5)

  • Figure 1: (a) The distribution of energy consumption of CPUs, GPUs, and other components for multiple HPC applications on a compute node from Aurora supercomputer. (b) Performance-energy trade-off in GPUs for the HPC application pot3d. At 1.6 GHz, energy consumption is 128.46 (kJ)=2.277 (kW)$\times$56.42 (s); at 1.1 GHz, energy consumption is 120.21 (kJ)=2.011 (kW)$\times$59.78 (s); at 0.8 GHz, energy consumption is 126.78 (kJ)=1.690 (kW)$\times$75.02 (s).
  • Figure 2: The architecture of an Aurora node.
  • Figure 3: Comparison of cumulative regret.
  • Figure 4: Switching cost analysis of EnergyUCB.
  • Figure 5: Impact of two reward functions and QoS analysis.