CO2-Meter: A Comprehensive Carbon Footprint Estimator for LLMs on Edge Devices

TL;DR

CO2-Meter addresses the rising carbon footprint of on-edge LLMs by jointly modeling operational and embodied emissions. It combines equation-based peripheral energy models, a GNN-based two-phase predictor for LLM inference energy, and a unit-level embodied carbon model for SoCs. The framework is validated against real measurements and demonstrated across use cases to identify carbon hotspots and guide sustainable edge deployments. The work enables carbon-aware design decisions and policy insights for greener AI at the edge.

Abstract

LLMs have transformed NLP, yet deploying them on edge devices poses great carbon challenges. Prior estimators remain incomplete, neglecting peripheral energy use, distinct prefill/decode behaviors, and SoC design complexity. This paper presents CO2-Meter, a unified framework for estimating operational and embodied carbon in LLM edge inference. Contributions include: (1) equation-based peripheral energy models and datasets; (2) a GNN-based predictor with phase-specific LLM energy data; (3) a unit-level embodied carbon model for SoC bottleneck analysis; and (4) validation showing superior accuracy over prior methods. Case studies show CO2-Meter's effectiveness in identifying carbon hotspots and guiding sustainable LLM design on edge platforms. Source code: https://github.com/fuzhenxiao/CO2-Meter

Figures (18)

• Figure 1: Autoregressive inference.
• Figure 2: A SoC die.
• Figure 3: The distinctive hardware characteristics of the prefill and decode phases in an LLM inference on rk3588.
• Figure 4: The LLM inference operational energy predictor of CO2-Meter.
• Figure 5: The power measurement infrastructure.