Computing Within Limits: An Empirical Study of Energy Consumption in ML Training and Inference

TL;DR

The paper investigates the environmental footprint of ML pipelines by conducting an empirical energy profiling study across multiple architectures and hardware configurations using software-based power measurements. It defines $E_{\mathrm{tr}}$ and $E_{\mathrm{in}}$ to quantify training and inference energy and analyzes how these relate to hardware utilisation and model characteristics, notably the MACs and MACs per parameter. The study finds that energy reductions can exceed marginal accuracy gains and highlights that inference workloads often drive substantial energy use, suggesting actionable guidelines for green MLOps, including short profiling to extrapolate total energy and power-capping strategies. The work provides a practical framework and open-source tooling to support energy-aware architecture and hyperparameter choices, contributing to sustainable ML deployment practices; the energy distribution across the MLOps pipeline is reported as roughly $31\%$ Data, $29\%$ Experimentation/Training, and $40\%$ Inference, with an inference-dominated split of $10:20:70$ across Experimentation, Training, and Inference in another benchmark.

Abstract

Machine learning (ML) has seen tremendous advancements, but its environmental footprint remains a concern. Acknowledging the growing environmental impact of ML this paper investigates Green ML, examining various model architectures and hyperparameters in both training and inference phases to identify energy-efficient practices. Our study leverages software-based power measurements for ease of replication across diverse configurations, models and datasets. In this paper, we examine multiple models and hardware configurations to identify correlations across the various measurements and metrics and key contributors to energy reduction. Our analysis offers practical guidelines for constructing sustainable ML operations, emphasising energy consumption and carbon footprint reductions while maintaining performance. As identified, short-lived profiling can quantify the long-term expected energy consumption. Moreover, model parameters can also be used to accurately estimate the expected total energy without the need for extensive experimentation.

Figures (7)

• Figure 1: ML model development and deployment phase and the associated MLOps life cycle.
• Figure 2: Training and inference duration (for $50$k samples).
• Figure 3: Average power usage with HC-2.
• Figure 4: Utilisation and power consumption (considering the GPU RAM usage) - HC-1.
• Figure 5: Loss, energy and accuracy per epoch, averaged across all models - the shaded areas show the range of values - HC-3.