Energy-Conscious LLM Decoding: Impact of Text Generation Strategies on GPU Energy Consumption

TL;DR

This work systematically quantifies how LLM decoding strategies shape GPU energy consumption during inference across translation, math, coding, and open-ended tasks. By benchmarking multiple deterministic and stochastic decoding methods on open-source models, the study reveals persistent energy-quality trade-offs and introduces energy-aware metrics like Efficiency Ratio. Hyperparameter sensitivity analyses show stochastic strategies often deliver robust quality with stable energy use, while beam-based methods achieve higher quality at greater energy cost. The results offer practical guidance for building energy-efficient LLM applications, underscoring that no single decoding method dominates all metrics and that strategy choice should align with task-specific priorities.

Abstract

Decoding strategies significantly influence the quality and diversity of the generated text in Large Language Models (LLMs), yet their impact on computational resources, particularly GPU energy consumption, is insufficiently studied. This paper investigates the relationship between text generation decoding techniques and energy efficiency, focusing on the trade-off between generation quality and GPU energy usage across diverse tasks and decoding configurations. By benchmarking multiple strategies across various tasks, including Translation, Math Problem Solving, Coding, and Open-ended text generation, we reveal how selecting appropriate decoding techniques with their tuned hyperparameters affects text quality and has measurable implications for energy consumption. Our findings show that the choice of decoding strategy can greatly impact GPU energy usage, even when it has a minimal effect on output quality. Different strategies also involve trade-offs between quality and energy efficiency, and no single decoding method is best in all cases across every metric. To the best of our knowledge, this is one of the first studies to examine decoding strategies in LLMs from the perspective of energy consumption, providing useful insights for building energy-efficient applications without compromising text generation quality.

Figures (2)

• Figure 1: Relative Standard Deviation of text generation quality and GPU energy usage across decoding strategies.
• Figure 2: Heatmap of Kendall’s Tau correlations between latency and energy consumption rankings of decoding strategies across different models and tasks.