Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Power Hungry Processing: Watts Driving the Cost of AI Deployment?

Alexandra Sasha Luccioni, Yacine Jernite, Emma Strubell

TL;DR

It is found that multi-purpose, generative architectures are orders of magnitude more expensive than task-specific systems for a variety of tasks, even when controlling for the number of model parameters.

Abstract

Recent years have seen a surge in the popularity of commercial AI products based on generative, multi-purpose AI systems promising a unified approach to building machine learning (ML) models into technology. However, this ambition of ``generality'' comes at a steep cost to the environment, given the amount of energy these systems require and the amount of carbon that they emit. In this work, we propose the first systematic comparison of the ongoing inference cost of various categories of ML systems, covering both task-specific (i.e. finetuned models that carry out a single task) and `general-purpose' models, (i.e. those trained for multiple tasks). We measure deployment cost as the amount of energy and carbon required to perform 1,000 inferences on representative benchmark dataset using these models. We find that multi-purpose, generative architectures are orders of magnitude more expensive than task-specific systems for a variety of tasks, even when controlling for the number of model parameters. We conclude with a discussion around the current trend of deploying multi-purpose generative ML systems, and caution that their utility should be more intentionally weighed against increased costs in terms of energy and emissions. All the data from our study can be accessed via an interactive demo to carry out further exploration and analysis.

Power Hungry Processing: Watts Driving the Cost of AI Deployment?

TL;DR

It is found that multi-purpose, generative architectures are orders of magnitude more expensive than task-specific systems for a variety of tasks, even when controlling for the number of model parameters.

Abstract

Recent years have seen a surge in the popularity of commercial AI products based on generative, multi-purpose AI systems promising a unified approach to building machine learning (ML) models into technology. However, this ambition of ``generality'' comes at a steep cost to the environment, given the amount of energy these systems require and the amount of carbon that they emit. In this work, we propose the first systematic comparison of the ongoing inference cost of various categories of ML systems, covering both task-specific (i.e. finetuned models that carry out a single task) and `general-purpose' models, (i.e. those trained for multiple tasks). We measure deployment cost as the amount of energy and carbon required to perform 1,000 inferences on representative benchmark dataset using these models. We find that multi-purpose, generative architectures are orders of magnitude more expensive than task-specific systems for a variety of tasks, even when controlling for the number of model parameters. We conclude with a discussion around the current trend of deploying multi-purpose generative ML systems, and caution that their utility should be more intentionally weighed against increased costs in terms of energy and emissions. All the data from our study can be accessed via an interactive demo to carry out further exploration and analysis.
Paper Structure (21 sections, 7 figures, 7 tables)

This paper contains 21 sections, 7 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Previous Work
  3. Methodology
  4. Task and dataset selection
  5. Models
  6. Task-specific Models.
  7. Multi-Purpose Models.
  8. Results
  9. Task-specific model analysis
  10. The environmental cost of multi-purpose systems
  11. Emissions of task-specific and multi-task architectures
  12. Differences within multi-purpose architectures
  13. Comparing model training and inference costs
  14. Discussion
  15. Generative tasks are more energy- and carbon-intensive compared to discriminative tasks
  16. ...and 6 more sections

Figures (7)

  • Figure 1: The 5 modalities examined in our study, with the number of parameters of each model on the x axis and the average amount of carbon emitted for 1000 inferences on the y axis. NB: Both axes are in logarithmic scale.
  • Figure 2: Model emissions (measured in g $CO_2eq$) and architecture type for each of the datasets from our analysis. The y axis is in logarithmic scale, dot size is proportional to model size.
  • Figure 3: Model size, measured in number of parameters (x axis, logarithmic scale) and text classification accuracy (y axis), with dot size indicating the quantity of emissions (logarithmic scale).
  • Figure 4: A plot of the total emissions (in grams of $CO_2eq$) for 1,000 inferences for all multi-purpose models.
  • Figure 5: A plot of the output length (X axis) and carbon emissions (Y axis) for the summarization task. The symbol refers to the type of architecture (BLOOMz vs Flan-T5), symbol size references the relative model size (in terms of the number of parameters), and color the input length.
  • ...and 2 more figures