AIMeter: Measuring, Analyzing, and Visualizing Energy and Carbon Footprint of AI Workloads

TL;DR

AIMeter addresses the fragmented landscape of energy and carbon measurement for AI workloads by integrating energy use, power draw, hardware metrics, and carbon emissions into a unified, time-series framework. It employs a three-layer architecture with high-frequency, multi-interface metric collection, standardized processing (including marginalized carbon intensity), and diverse demonstration modes for reporting and visualization. The paper demonstrates the approach with a Llama2-7b inference on an NVIDIA A800, revealing clear phase-dependent energy dynamics and geographic differences in carbon emissions, while showing low overhead for practical use. Overall, AIMeter advances reproducible, carbon-aware benchmarking and supports Green AI practices by providing a practical, extensible toolkit and open-source codebase at https://github.com/SusCom-Lab/AIMeter.

Abstract

The rapid advancement of AI, particularly large language models (LLMs), has raised significant concerns about the energy use and carbon emissions associated with model training and inference. However, existing tools for measuring and reporting such impacts are often fragmented, lacking systematic metric integration and offering limited support for correlation analysis among them. This paper presents AIMeter, a comprehensive software toolkit for the measurement, analysis, and visualization of energy use, power draw, hardware performance, and carbon emissions across AI workloads. By seamlessly integrating with existing AI frameworks, AIMeter offers standardized reports and exports fine-grained time-series data to support benchmarking and reproducibility in a lightweight manner. It further enables in-depth correlation analysis between hardware metrics and model performance and thus facilitates bottleneck identification and performance enhancement. By addressing critical limitations in existing tools, AIMeter encourages the research community to weigh environmental impact alongside raw performance of AI workloads and advances the shift toward more sustainable "Green AI" practices. The code is available at https://github.com/SusCom-Lab/AIMeter.

Figures (7)

• Figure 1: The architecture of AIMeter.
• Figure 2: Power and frequency during inference phase.
• Figure 3: Shifting bottlenecks between phases.
• Figure 4: Occupancy and active during decode phase.
• Figure 5: Carbon emissions from the same AI task in two neighboring provinces of Canada, respectively.