Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Harnessing Generative Pre-Trained Transformer for Datacenter Packet Trace Generation

Chen Griner

TL;DR

The paper tackles the lack of realistic datacenter traffic traces and privacy concerns by proposing DTG-GPT, a GPT-based packet-trace generator trained on a small set of traces. The model uses a multi-embedding input with a meta-data embedding that encodes segment, field, and trace identifiers to model deterministic packet headers and temporal segmentation. Evaluation across traffic matrices, trace complexity, bursts, and novelty shows DTG-GPT can synthesize traces with similar spatiotemporal structure and high novelty, across HPC and Facebook-domain traces, at multiple scales. This work suggests a path toward privacy-preserving trace sharing by releasing trained models or weights instead of raw traces, enabling researchers to study and optimize future datacenters.

Abstract

Today, the rapid growth of applications reliant on datacenters calls for new advancements to meet the increasing traffic and computational demands. Traffic traces from datacenters are essential for further development and optimization of future datacenters. However, traces are rarely released to the public. Researchers often use simplified mathematical models that lack the depth needed to recreate intricate traffic patterns and, thus, miss optimization opportunities found in realistic traffic. In this preliminary work, we introduce DTG-GPT, a packet-level Datacenter Traffic Generator (DTG), based on the generative pre-trained transformer (GPT) architecture used by many state-of-the-art large language models. We train our model on a small set of available traffic traces from different domains and offer a simple methodology to evaluate the fidelity of the generated traces to their original counterparts. We show that DTG-GPT can synthesize novel traces that mimic the spatiotemporal patterns found in real traffic traces. We further demonstrate that DTG-GPT can generate traces for networks of different scales while maintaining fidelity. Our findings indicate the potential that, in the future, similar models to DTG-GPT will allow datacenter operators to release traffic information to the research community via trained GPT models.

Harnessing Generative Pre-Trained Transformer for Datacenter Packet Trace Generation

TL;DR

The paper tackles the lack of realistic datacenter traffic traces and privacy concerns by proposing DTG-GPT, a GPT-based packet-trace generator trained on a small set of traces. The model uses a multi-embedding input with a meta-data embedding that encodes segment, field, and trace identifiers to model deterministic packet headers and temporal segmentation. Evaluation across traffic matrices, trace complexity, bursts, and novelty shows DTG-GPT can synthesize traces with similar spatiotemporal structure and high novelty, across HPC and Facebook-domain traces, at multiple scales. This work suggests a path toward privacy-preserving trace sharing by releasing trained models or weights instead of raw traces, enabling researchers to study and optimize future datacenters.

Abstract

Today, the rapid growth of applications reliant on datacenters calls for new advancements to meet the increasing traffic and computational demands. Traffic traces from datacenters are essential for further development and optimization of future datacenters. However, traces are rarely released to the public. Researchers often use simplified mathematical models that lack the depth needed to recreate intricate traffic patterns and, thus, miss optimization opportunities found in realistic traffic. In this preliminary work, we introduce DTG-GPT, a packet-level Datacenter Traffic Generator (DTG), based on the generative pre-trained transformer (GPT) architecture used by many state-of-the-art large language models. We train our model on a small set of available traffic traces from different domains and offer a simple methodology to evaluate the fidelity of the generated traces to their original counterparts. We show that DTG-GPT can synthesize novel traces that mimic the spatiotemporal patterns found in real traffic traces. We further demonstrate that DTG-GPT can generate traces for networks of different scales while maintaining fidelity. Our findings indicate the potential that, in the future, similar models to DTG-GPT will allow datacenter operators to release traffic information to the research community via trained GPT models.
Paper Structure (18 sections, 4 equations, 13 figures, 1 table)

This paper contains 18 sections, 4 equations, 13 figures, 1 table.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Traffic & Trace models
  4. Trace Complexity
  5. DTG-GPT Architecture
  6. Training and Generation Setup
  7. Data
  8. Fine-Tuning
  9. Tokenization
  10. Training Setup
  11. Trace Generation
  12. Evaluation
  13. Traffic Matrices
  14. Trace Complexity
  15. Traffic Bursts
  16. ...and 3 more sections

Figures (13)

  • Figure 1: Two traffic matrices of $8M$ requests. Where (a) is a real trace from an HPC cluster (NeckBone) doe2016characterization and (b) is a counterpart trace generated with DTG-GPT. The maximal element was clipped to $100$ requests to enhance contrast.
  • Figure 2: Simplified scheme for our DTG-GPT model .
  • Figure 3: Traffic matrices for several of the communication traces. Colors are scaled individually, and the scale is provided at the top of each matrix. Axes represent source IDs (vertical) and destination IDs (horizontal).
  • Figure 4: Original WEB trace traffic matrix, along with generated traces using two different temperatures.
  • Figure 5: The complexity map of seven real traces and their generated counterparts. An opaque circle designates original ($O$) traces, while generated ($G$) traces have a more transparent circle. (a) Represents traces from the HPC set. (b) Represents traces from the Facebook set.
  • ...and 8 more figures