Super-resolution on network telemetry time series

TL;DR

This work presents Zoom2Net, a transformer-based model for network imputation that incorporates domain knowledge through operational and measurement constraints, ensuring that the imputed network telemetry time series are not only realistic but also align with existing measurements and are plausible.

Abstract

Fine-grained monitoring is crucial for multiple data-driven tasks such as debugging, provisioning, and securing networks. Yet, practical constraints in collecting, extracting, and storing data often force operators to use coarse-grained sampled monitoring, degrading the performance of the various tasks. In this work, we explore the feasibility of leveraging the correlations among coarse-grained time series to impute their fine-grained counterparts in software. We present Zoom2Net, a transformer-based model for network imputation that incorporates domain knowledge through operational and measurement constraints, ensuring that the imputed network telemetry time series are not only realistic but also align with existing measurements and are plausible. This approach enhances the capabilities of current monitoring infrastructures, allowing operators to gain more insights into system behaviors without the need for hardware upgrades. We evaluate Zoom2Net on four diverse datasets (e.g. cloud telemetry and Internet data transfer) and use cases (such as bursts analysis and traffic classification). We demonstrate that Zoom2Net consistently achieves high imputation accuracy with a zoom-in factor of up to 100 and performs better on downstream tasks compared to baselines by an average of 38%.

Figures (11)

• Figure 1: Fine-grained signals $T_{r}$ of a networked system are sampled by monitoring tools, resulting in coarse-grained time series $T_{s}$ available to operators. In practice, $T_{r}$ is only available for training and is collected over a short period using specialized hardware or traffic mirroring.
• Figure 2: Zoom2Net takes a set of coarse-grained time series $T_{s}$ as input and outputs imputed fine-grained time series $\hat{T}_{r}$ which is fed to multiple downstream tasks.
• Figure 3: Two distinct fine-grained queue length behaviors result in the same sampled maximum queue length. But they can be distinguished from different sampled packet drops and sent counts.
• Figure 4: Two distinct fine-grained queue length encompass almost identical coarse-grained signals. A transformer model trained with MSE, having seen both (and more), would generate an average (green line), obfuscating the burst.
• Figure 5: Ground truth fine-grained queue length at 1ms (blue) and imputed fine-grained queue length at 1ms (orange). A plain transformer catches trends but outputs results that are inconsistent with maximum and periodic samples.