Dissecting Transformers: A CLEAR Perspective towards Green AI

TL;DR

Large language models incur substantial energy costs during inference, yet prior work provides only coarse, model-level metrics. CLEAR introduces a component-level energy assessment method that caches per-component activations and uses an amplification strategy to overcome coarse sensor granularity, enabling precise energy attribution across Transformer blocks. The study finds that attention blocks exhibit the highest energy per FLOP and that total energy follows a two-term model $E(L) \approx E_0 + k \cdot \mathrm{FLOPs}(L)$ with component-dependent $k$, while %Capture remains above 90% across architectures. These insights enable targeted architectural and hardware–software co-design for Green AI, moving energy considerations from afterthoughts to integral design criteria.

Abstract

The rapid adoption of Large Language Models (LLMs) has raised significant environmental concerns. Unlike the one-time cost of training, LLM inference occurs continuously at a global scale and now dominates the AI energy footprint. Yet, most sustainability studies report only coarse, model-level metrics due to the lack of fine-grained measurement methods, treating energy efficiency more as an afterthought than as a primary objective. We present the first fine-grained empirical analysis of inference energy across core components of transformer architecture. We propose a novel methodology, Component-Level Energy Assessment via Repeated sampling (CLEAR), to overcome temporal mismatch between microsecond scale component execution and monitoring of millisecond (ms) scale energy sensors. Using CLEAR, we evaluate 15 models spanning four distinct architecture types and consistently keep component-wise energy variance below 9.5\% while capturing more than 90\% of the model's total energy as individual components. Our empirical analysis reveals that Attention blocks consume significantly more energy per floating-point operation (FLOP), indicating that energy consumption is not proportionally aligned with FLOP counts. This shows that FLOPs alone fail to capture the true energy cost at a component level. Our findings establish detailed component-level energy baselines and provide insight as an initial step to build energy-efficient transformer models through component-level optimizations.

Figures (8)

• Figure 1: CLEAR pipeline: (1) Store forward-pass activations in Activation Store, (2) Measure per-component energy consumption by isolating each component (e.g., Attention Block) and replaying stored activations, (3) Validate for consistency and completeness, and (4) perform Empirical energy analysis across models.
• Figure 2:
• Figure 3: E/FLOP ratio for model components in Qwen-2.5-3B (left) and Gemma-3-4B (right). Attention consistently shows highest E/FLOP ratio across all input lengths.
• Figure 4: Qwen-2.5-3B: $E/\text{FLOP}$(Row-1) and $\Delta E/\Delta \text{FLOP}$(Row-2). Gemma-3-4B: $E/\text{FLOP}$(Row-3) and $\Delta E/\Delta \text{FLOP}$(Row-4). For both models, the $E/\text{FLOP}$ ratio decreases with input length, while marginal energy per FLOP remains nearly constant.
• Figure 5: Standard Deviation across different Energy measurements. We observe a decrease in Standard Deviation in our Energy Measurement Approach with increasing Energy Values.