Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PET: Multi-agent Independent PPO-based Automatic ECN Tuning for High-Speed Data Center Networks

Kai Cheng, Ting Wang, Xiao Du, Shuyi Du, Haibin Cai

TL;DR

PET targets automatic ECN threshold tuning in high-speed data center networks under incast and mixed mice/elephant traffic. It uses a multi-agent IPPO-based DRL approach under a DTDE framework, incorporating six congestion-driving metrics to learn ECN configurations with offline+online hybrid training for fast adaptation on commodity switches. The design avoids global experience replay to reduce memory and bandwidth overhead while maintaining robustness to nonstationary traffic patterns. Empirical results in ns-3 show PET outperforms static schemes and ACC in flow completion time, latency, convergence speed, and robustness to failures and pattern shifts.

Abstract

Explicit Congestion Notification (ECN)-based congestion control schemes have been widely adopted in high-speed data center networks (DCNs), where the ECN marking threshold plays a determinant role in guaranteeing a packet lossless DCN. However, existing approaches either employ static settings with immutable thresholds that cannot be dynamically self-adjusted to adapt to network dynamics, or fail to take into account many-to-one traffic patterns and different requirements of different types of traffic, resulting in relatively poor performance. To address these problems, this paper proposes a novel learning-based automatic ECN tuning scheme, named PET, based on the multi-agent Independent Proximal Policy Optimization (IPPO) algorithm. PET dynamically adjusts ECN thresholds by fully considering pivotal congestion-contributing factors, including queue length, output data rate, output rate of ECN-marked packets, current ECN threshold, the extent of incast, and the ratio of mice and elephant flows. PET adopts the Decentralized Training and Decentralized Execution (DTDE) paradigm and combines offline and online training to accommodate network dynamics. PET is also fair and readily deployable with commodity hardware. Comprehensive experimental results demonstrate that, compared with state-of-the-art static schemes and the learning-based automatic scheme, our PET achieves better performance in terms of flow completion time, convergence rate, queue length variance, and system robustness.

PET: Multi-agent Independent PPO-based Automatic ECN Tuning for High-Speed Data Center Networks

TL;DR

PET targets automatic ECN threshold tuning in high-speed data center networks under incast and mixed mice/elephant traffic. It uses a multi-agent IPPO-based DRL approach under a DTDE framework, incorporating six congestion-driving metrics to learn ECN configurations with offline+online hybrid training for fast adaptation on commodity switches. The design avoids global experience replay to reduce memory and bandwidth overhead while maintaining robustness to nonstationary traffic patterns. Empirical results in ns-3 show PET outperforms static schemes and ACC in flow completion time, latency, convergence speed, and robustness to failures and pattern shifts.

Abstract

Explicit Congestion Notification (ECN)-based congestion control schemes have been widely adopted in high-speed data center networks (DCNs), where the ECN marking threshold plays a determinant role in guaranteeing a packet lossless DCN. However, existing approaches either employ static settings with immutable thresholds that cannot be dynamically self-adjusted to adapt to network dynamics, or fail to take into account many-to-one traffic patterns and different requirements of different types of traffic, resulting in relatively poor performance. To address these problems, this paper proposes a novel learning-based automatic ECN tuning scheme, named PET, based on the multi-agent Independent Proximal Policy Optimization (IPPO) algorithm. PET dynamically adjusts ECN thresholds by fully considering pivotal congestion-contributing factors, including queue length, output data rate, output rate of ECN-marked packets, current ECN threshold, the extent of incast, and the ratio of mice and elephant flows. PET adopts the Decentralized Training and Decentralized Execution (DTDE) paradigm and combines offline and online training to accommodate network dynamics. PET is also fair and readily deployable with commodity hardware. Comprehensive experimental results demonstrate that, compared with state-of-the-art static schemes and the learning-based automatic scheme, our PET achieves better performance in terms of flow completion time, convergence rate, queue length variance, and system robustness.
Paper Structure (29 sections, 13 equations, 5 figures, 1 algorithm)

This paper contains 29 sections, 13 equations, 5 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Introduction
  3. Related Work
  4. Static ECN Schemes
  5. Dynamic ECN Tuning Schemes
  6. Learning-based Automatic ECN Tuning Schemes
  7. Preliminaries
  8. Explicit Congestion Notification
  9. Incast Problem
  10. Design Goals
  11. PET Approach Design
  12. Design Overview
  13. Reinforcement Learning Modeling
  14. Implementation Choices
  15. Problem Formalization
  16. ...and 14 more sections

Figures (5)

  • Figure 1: Reinforcement learning modeling
  • Figure 2: Overview of PET architecture
  • Figure 3: Traffic distributions
  • Figure : PET's Learning Algorithm based on Multi-agent IPPO
  • Figure :