Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Carbon-Aware End-to-End Data Movement

Jacob Goldverg, Hasibul Jamil, Elvis Rodriguez, Tevfik Kosar

TL;DR

The paper tackles the rising carbon footprint of end-to-end data movement in cloud, edge, and AI workloads. It develops a measurement framework that aggregates end-system emissions and hop-by-hop network carbon to yield end-to-end carbon intensity. It introduces three carbon-aware strategies—shifting transfers in time, shifting in space via alternative data sources, and overlay networks with FTNs—and defines a practical metric $carbonScore = \\frac{bytes}{CI \\times duration}$ implemented in the Pmeter tool. Experiments on Chameleon Cloud and a MacBook demonstrate significant potential savings (up to ~2x) and highlight deployment implications for cloud providers, CDNs, and SLA-driven green computing.

Abstract

The latest trends in the adoption of cloud, edge, and distributed computing, as well as a rise in applying AI/ML workloads, have created a need to measure, monitor, and reduce the carbon emissions of these compute-intensive workloads and the associated communication costs. The data movement over networks has considerable carbon emission that has been neglected due to the difficulty in measuring the carbon footprint of a given end-to-end network path. We present a novel network carbon footprint measuring mechanism and propose three ways in which users can optimize scheduling network-intensive tasks to enable carbon savings through shifting tasks in time, space, and overlay networks based on the geographic carbon intensity.

Carbon-Aware End-to-End Data Movement

TL;DR

The paper tackles the rising carbon footprint of end-to-end data movement in cloud, edge, and AI workloads. It develops a measurement framework that aggregates end-system emissions and hop-by-hop network carbon to yield end-to-end carbon intensity. It introduces three carbon-aware strategies—shifting transfers in time, shifting in space via alternative data sources, and overlay networks with FTNs—and defines a practical metric implemented in the Pmeter tool. Experiments on Chameleon Cloud and a MacBook demonstrate significant potential savings (up to ~2x) and highlight deployment implications for cloud providers, CDNs, and SLA-driven green computing.

Abstract

The latest trends in the adoption of cloud, edge, and distributed computing, as well as a rise in applying AI/ML workloads, have created a need to measure, monitor, and reduce the carbon emissions of these compute-intensive workloads and the associated communication costs. The data movement over networks has considerable carbon emission that has been neglected due to the difficulty in measuring the carbon footprint of a given end-to-end network path. We present a novel network carbon footprint measuring mechanism and propose three ways in which users can optimize scheduling network-intensive tasks to enable carbon savings through shifting tasks in time, space, and overlay networks based on the geographic carbon intensity.
Paper Structure (12 sections, 1 equation, 5 figures, 2 tables)

This paper contains 12 sections, 1 equation, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Carbon Intensity of Data Path
  4. End-system Monitoring
  5. Discovering a Network Path
  6. Geolocating and Measuring the Carbon Intensity of a Network Path
  7. Carbon Intensity of a File Transfer
  8. Experimentation
  9. Shifting in Time
  10. Shifting in Space
  11. Using an Overlay Network of File Transfer Nodes
  12. Future Work and Conclusion

Figures (5)

  • Figure 1: Energy consumption share of the end systems vs network infrastructure during end-to-end data transfer. This ratio depends on the number of network devices (i.e., routers, switches, etc.) between the end systems, and how much power each device consumes.
  • Figure 2: Carbon Intensity of IP addresses from UC to TACC
  • Figure 3: Average Carbon Intensity at 1-Hour Intervals from UC to TACC, April 14-16, 2024
  • Figure 4: Carbon Index of 10 states in the US
  • Figure 5: Measuring the Carbon Intensity from UC and M1 to TACC