OpenCarbonEval: A Unified Carbon Emission Estimation Framework in Large-Scale AI Models

TL;DR

OpenCarbonEval introduces a unified cross-modal framework that uses Little's Law and Dynamic Throughput Modeling to predict training carbon emissions before initiating large-scale model training. By separating Operational and Embodied Carbon and modeling hardware throughput with a two-stage process, it achieves superior predictive accuracy over static baselines across language and vision tasks. The approach relies on $f(t) = \ln(1 + \alpha t)$ and the emissions relation $\mathcal{C}(P,D) = \int_{0}^{T} f(t) dt$ to connect compute, time, and carbon, while accounting for hardware-gen effects via the Equipment Era Effect Law. This work enables carbon-aware planning for AI development and deployment, promoting more sustainable practices in the AI community.

Abstract

In recent years, large-scale auto-regressive models have made significant progress in various tasks, such as text or video generation. However, the environmental impact of these models has been largely overlooked, with a lack of assessment and analysis of their carbon footprint. To address this gap, we introduce OpenCarbonEval, a unified framework for integrating large-scale models across diverse modalities to predict carbon emissions, which could provide AI service providers and users with a means to estimate emissions beforehand and help mitigate the environmental pressure associated with these models. In OpenCarbonEval, we propose a dynamic throughput modeling approach that could capture workload and hardware fluctuations in the training process for more precise emissions estimates. Our evaluation results demonstrate that OpenCarbonEval can more accurately predict training emissions than previous methods, and can be seamlessly applied to different modal tasks. Specifically, we show that OpenCarbonEval achieves superior performance in predicting carbon emissions for both visual models and language models. By promoting sustainable AI development and deployment, OpenCarbonEval can help reduce the environmental impact of large-scale models and contribute to a more environmentally responsible future for the AI community.

Figures (7)

• Figure 1: Large-scale models' environmental impact covering 42 large-scale AI models across 15 tasks. OpenCarbonEval enables the estimation of carbon emissions for various models, facilitating a more sustainable AI development process.
• Figure 2: The overall pipeline of OpenCarbonEval. OpenCarbonEval leverages Little's Law to establish a rigorous framework for modeling computation, throughput, and time, thereby enabling the precise estimation of carbon emissions associated with the model.
• Figure 3: The two-stage modeling about hardware performance. The cold starting phase is characterized by a brief duration of $t_1$, after which the actual training commences. Subsequently, the entire training process is completed within a total time span of $t_2$.
• Figure 4: The data points indicate the model under different hardware. In our experimental setup, we aggregated hardware devices sharing a common prefix, such as Nvidia A100, into a single category to facilitate analysis and comparison.
• Figure 5: Distribution about different hardware. The throughput-$\alpha$ distribution is centered around the mean, exhibiting a pronounced clustering of values in close proximity to the average, thereby underscoring the stability of our model's performance.