Magneton: Optimizing Energy Efficiency of ML Systems via Differential Energy Debugging

TL;DR

ML energy efficiency remains hampered by software-level waste overlooked by hardware-focused optimizations. The authors propose differential energy debugging and Magneton, an operator-centric profiler that compares semantically equivalent subgraphs across ML systems to locate energy hotspots and diagnose root causes. Through operator-level tracing, tensor-based semantic matching, and static analysis, Magneton identifies known and unknown energy inefficiencies with low overhead. Evaluations across nine systems show substantial energy waste reductions and the discovery of previously unseen issues, demonstrating practical impact for reducing energy in ML pipelines.

Abstract

The training and deployment of machine learning (ML) models have become extremely energy-intensive. While existing optimization efforts focus primarily on hardware energy efficiency, a significant but overlooked source of inefficiency is software energy waste caused by poor software design. This often includes redundant or poorly designed operations that consume more energy without improving performance. These inefficiencies arise in widely used ML frameworks and applications, yet developers often lack the visibility and tools to detect and diagnose them. We propose differential energy debugging, a novel approach that leverages the observation that competing ML systems often implement similar functionality with vastly different energy consumption. Building on this insight, we design and implement Magneton, an energy profiler that compares energy consumption between similar ML systems at the operator level and automatically pinpoints code regions and configuration choices responsible for excessive energy use. Applied to 9 popular ML systems spanning LLM inference, general ML frameworks, and image generation, Magneton detects and diagnoses 16 known cases of software energy inefficiency and further discovers 8 previously unknown cases, 7 of which have been confirmed by developers.

Figures (10)

• Figure 1: Software energy waste, performance issues, and performance-energy trade-offs in the design space.
• Figure 2: Total energy consumption and breakdowns of top 5 operators in HuggingFace Transformers.
• Figure 3: Code snippet from HuggingFace related to the energy inefficiency issue.
• Figure 4: The power consumption of the join and early exit in DDP manager.
• Figure 5: (a) Summary of popular machine learning repositories grouped by category. (b) Energy consumption per token of different LLM inference systems during offline inference, $(x, y)$ means each request contains $x$ input and $y$ output tokens. (c) Energy consumption of the convolution operator in different ML libraries. (d) Energy consumption per image patch of different image generation systems.