Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bridging the Gap Between Simulated and Real Network Data Using Transfer Learning

Carlos Güemes-Palau, Miquel Ferriol-Galmés, Jordi Paillisse-Vilanova, Albert López-Brescó, Pere Barlet-Ros, Albert Cabellos-Aparicio

TL;DR

The paper addresses the data scarcity and domain shift problem confronted by ML-based network performance models trained on simulated data but deployed in real networks. It introduces a transfer-learning pipeline that pre-trains RouteNet-Fermi on abundant simulated data and then fine-tunes the model with a small real-world dataset, exploring both manual and automated weight-transfer strategies. Experiments with OMNeT++ simulations and a real testbed show substantial improvements in predictive accuracy, with up to 88% MAPE reduction for Poisson traffic, 80% for On/Off, and 48% for MAWI when using 50 real scenarios (37% with only 10 scenarios). The study demonstrates data-efficient adaptation of network predictors to real environments and provides practical guidance on selecting fine-tuning configurations, thereby reducing reliance on large real-world datasets while preserving performance.

Abstract

Machine Learning (ML)-based network models provide fast and accurate predictions for complex network behaviors but require substantial training data. Collecting such data from real networks is often costly and limited, especially for critical scenarios like failures. As a result, researchers commonly rely on simulated data, which reduces accuracy when models are deployed in real environments. We propose a hybrid approach leveraging transfer learning to combine simulated and real-world data. Using RouteNet-Fermi, we show that fine-tuning a pre-trained model with a small real dataset significantly improves performance. Our experiments with OMNeT++ and a custom testbed reduce the Mean Absolute Percentage Error (MAPE) in packet delay prediction by up to 88%. With just 10 real scenarios, MAPE drops by 37%, and with 50 scenarios, by 48%.

Bridging the Gap Between Simulated and Real Network Data Using Transfer Learning

TL;DR

The paper addresses the data scarcity and domain shift problem confronted by ML-based network performance models trained on simulated data but deployed in real networks. It introduces a transfer-learning pipeline that pre-trains RouteNet-Fermi on abundant simulated data and then fine-tunes the model with a small real-world dataset, exploring both manual and automated weight-transfer strategies. Experiments with OMNeT++ simulations and a real testbed show substantial improvements in predictive accuracy, with up to 88% MAPE reduction for Poisson traffic, 80% for On/Off, and 48% for MAWI when using 50 real scenarios (37% with only 10 scenarios). The study demonstrates data-efficient adaptation of network predictors to real environments and provides practical guidance on selecting fine-tuning configurations, thereby reducing reliance on large real-world datasets while preserving performance.

Abstract

Machine Learning (ML)-based network models provide fast and accurate predictions for complex network behaviors but require substantial training data. Collecting such data from real networks is often costly and limited, especially for critical scenarios like failures. As a result, researchers commonly rely on simulated data, which reduces accuracy when models are deployed in real environments. We propose a hybrid approach leveraging transfer learning to combine simulated and real-world data. Using RouteNet-Fermi, we show that fine-tuning a pre-trained model with a small real dataset significantly improves performance. Our experiments with OMNeT++ and a custom testbed reduce the Mean Absolute Percentage Error (MAPE) in packet delay prediction by up to 88%. With just 10 real scenarios, MAPE drops by 37%, and with 50 scenarios, by 48%.

Paper Structure

This paper contains 17 sections, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Transfer Learning
  4. RouteNet-Fermi
  5. Problem statement
  6. Methodology and experimental design
  7. Model architecture
  8. Manual transfer learning
  9. Automated transfer learning
  10. Testbed
  11. Evaluation
  12. Fine-tuning the network model
  13. Impact of real-world network data availability
  14. Related work
  15. Network modeling
  16. ...and 2 more sections

Figures (5)

  • Figure 1: Summary of the proposed hybrid approach. Simulated network scenarios are used to first train a network model. Then, this model is fine-tuned using a smaller dataset from real-world network data.
  • Figure 2: Visual example of fine-tuning a RouteNet-Fermi ferriolgalmés2022routenetfermi model, where the Encoding is frozen, the MPA is fine-tuned, and the Readout is re-trained.
  • Figure 3: Diagram summarizing the testbed's structure.
  • Figure 4: Probability Density Function (PDF) of percentage error when predicting the mean packet delay of each flow-window pair. It compares the best manual fine-tuning configuration (freezing Encoding, fine-tuning MPA, re-training Readout) against the real-world network data only model.
  • Figure 5: Impact of real-world network data availability on the resulting model's MAPE when predicting the average packet delay in each flow-window pair. Network scenarios belong to the MAWI distribution.