AI-based Dynamic Schedule Calculation in Time Sensitive Networks using GCN-TD3

TL;DR

The paper tackles dynamic flow scheduling in Time-Sensitive Networking by introducing GCN-TD3, an AI-driven runtime scheduler that leverages graph convolutional representations of TSN topology and TD3 reinforcement learning to adaptively admit and place arriving TT flows while preserving existing schedules. An ILP-based offline scheduler provides an initial and fallback schedule, and OMNeT++/INET extensions enable realistic dynamic simulation. Empirical results show GCN-TD3 achieving around 90% TT-flow admission within a fixed GCL length and reducing jitter to approximately $2\ \mu s$, outperforming DDQN and DDPG baselines and converging within about 4000 training iterations. The work demonstrates a scalable, topology-aware dynamic scheduling framework with practical impact for industrial TSN deployments, while highlighting areas for future work such as dynamic path selection and real-world validation with a digital twin.

Abstract

Offline scheduling in Time Sensitive Networking (TSN) utilizing the Time Aware Shaper (TAS) facilitates optimal deterministic latency and jitter-bounds calculation for Time- Triggered (TT) flows. However, the dynamic nature of traffic in industrial settings necessitates a strategy for adaptively scheduling flows without interrupting existing schedules. Our research identifies critical gaps in current dynamic scheduling methods for TSN and introduces the novel GCN-TD3 approach. This novel approach utilizes a Graph Convolutional Network (GCN) for representing the various relations within different components of TSN and employs the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm to dynamically schedule any incoming flow. Additionally, an Integer Linear Programming (ILP) based offline scheduler is used both to initiate the simulation and serve as a fallback mechanism. This mechanism is triggered to recalculate the entire schedule when the predefined threshold of Gate Control List(GCL) length exceeds. Comparative analyses demonstrate that GCN-TD3 outperforms existing methods like Deep Double Q-Network (DDQN) and Deep Deterministic Policy Gradient (DDPG), exhibiting convergence within 4000 epochs with a 90\% dynamic TT flow admission rate while maintaining deadlines and reducing jitter to as low as 2us. Finally, two modules were developed for the OMNeT++ simulator, facilitating dynamic simulation to evaluate the methodology.

Figures (6)

• Figure 1: (a) The proposed node model. (b) Pictorial representation of data path and network data path of 4 flows.
• Figure 2: GCN-TD3 Framework overview.
• Figure 3: Process of fixed size representation of variable size topology and flow features by GCN.
• Figure 4: No. of Scheduled Flows over 5000 training steps.
• Figure 5: Flow admission rate over 5000 training steps.