Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the Power Saving in High-Speed Ethernet-based Networks for Supercomputers and Data Centers

Miguel Sánchez de la Rosa, Francisco J. andújar, Jesus Escudero-Sahuquillo, José L. Sánchez, Francisco J. Alfaro-Cortés

TL;DR

This paper addresses the growing power demands of HPC and data-center interconnects by evaluating Energy Efficient Ethernet (EEE) based power management and extending PerfBound with PerfBoundCorrect to bound degradation. It proposes a simulation-driven methodology on BXIv3-inspired networks to co-design hardware-enabled LPI/EEE transitions with application-aware traffic, using per-port inactivity histograms and hop-distance corrections. Across workloads including LAMMPS, PATMOS, MLWF, and AlexNet, the authors show that energy savings on links can exceed 5% with minimal performance penalties, and that PerfBoundCorrect consistently reduces latency and improves energy efficiency compared to the original PerfBound. The results indicate practical, portable strategies for energy-proportional HPC interconnects and lay groundwork for future Ethernet-derived HPC interconnects such as BXIv4/Omnipath, contributing to lower operating costs and greener large-scale systems. $t_{PDT}$-based gating combined with robust prediction corrections emerges as a key lever for achieving energy savings with controlled performance impact.

Abstract

The increase in computation and storage has led to a significant growth in the scale of systems powering applications and services, raising concerns about sustainability and operational costs. In this paper, we explore power-saving techniques in high-performance computing (HPC) and datacenter networks, and their relation with performance degradation. From this premise, we propose leveraging Energy Efficient Ethernet (EEE), with the flexibility to extend to conventional Ethernet or upcoming Ethernet-derived interconnect versions of BXI and Omnipath. We analyze the PerfBound proposal, identifying possible improvements and modeling it into a simulation framework. Through different experiments, we examine its impact on performance and determine the most appropriate interconnect. We also study traffic patterns generated by selected HPC and machine learning applications to evaluate the behavior of power-saving techniques. From these experiments, we provide an analysis of how applications affect system and network energy consumption. Based on this, we disclose the weakness of dynamic power-down mechanisms and propose an approach that improves energy reduction with minimal or no performance penalty. To our knowledge, this is the first power management proposal tailored to future Ethernet-based HPC architectures, with promising results.

On the Power Saving in High-Speed Ethernet-based Networks for Supercomputers and Data Centers

TL;DR

This paper addresses the growing power demands of HPC and data-center interconnects by evaluating Energy Efficient Ethernet (EEE) based power management and extending PerfBound with PerfBoundCorrect to bound degradation. It proposes a simulation-driven methodology on BXIv3-inspired networks to co-design hardware-enabled LPI/EEE transitions with application-aware traffic, using per-port inactivity histograms and hop-distance corrections. Across workloads including LAMMPS, PATMOS, MLWF, and AlexNet, the authors show that energy savings on links can exceed 5% with minimal performance penalties, and that PerfBoundCorrect consistently reduces latency and improves energy efficiency compared to the original PerfBound. The results indicate practical, portable strategies for energy-proportional HPC interconnects and lay groundwork for future Ethernet-derived HPC interconnects such as BXIv4/Omnipath, contributing to lower operating costs and greener large-scale systems. -based gating combined with robust prediction corrections emerges as a key lever for achieving energy savings with controlled performance impact.

Abstract

The increase in computation and storage has led to a significant growth in the scale of systems powering applications and services, raising concerns about sustainability and operational costs. In this paper, we explore power-saving techniques in high-performance computing (HPC) and datacenter networks, and their relation with performance degradation. From this premise, we propose leveraging Energy Efficient Ethernet (EEE), with the flexibility to extend to conventional Ethernet or upcoming Ethernet-derived interconnect versions of BXI and Omnipath. We analyze the PerfBound proposal, identifying possible improvements and modeling it into a simulation framework. Through different experiments, we examine its impact on performance and determine the most appropriate interconnect. We also study traffic patterns generated by selected HPC and machine learning applications to evaluate the behavior of power-saving techniques. From these experiments, we provide an analysis of how applications affect system and network energy consumption. Based on this, we disclose the weakness of dynamic power-down mechanisms and propose an approach that improves energy reduction with minimal or no performance penalty. To our knowledge, this is the first power management proposal tailored to future Ethernet-based HPC architectures, with promising results.
Paper Structure (32 sections, 2 equations, 17 figures, 6 tables)

This paper contains 32 sections, 2 equations, 17 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Problem Statement
  3. Background
  4. Energy consumption in high-speed interconnection networks
  5. Initial attempts for reducing link power consumption
  6. Low Power Idle (LPI)
  7. Energy Efficient Ethernet (EEE)
  8. Proposals to complement EEE
  9. Frame Buffering
  10. Power-down Threshold
  11. PerfBound
  12. BXIv3 network technology
  13. Power consumption model
  14. Link power-levels and transition logic
  15. PerfBound implementation details
  16. ...and 17 more sections

Figures (17)

  • Figure 1: Inactivity histogram for a port when executing different applications.
  • Figure 2: Port switching between Wake and Sleep states due to packet transmissions.
  • Figure 3: Visual representation of Deep Sleep and Fast Wake power states.
  • Figure 4: Diagram of LPI using PDT.
  • Figure 5: Port state synchronization between two switch ports.
  • ...and 12 more figures