Lyapunov-guided Multi-Agent Reinforcement Learning for Delay-Sensitive Wireless Scheduling

TL;DR

A two-stage intelligent scheduler is proposed to minimize the packet-level delay jitter while guaranteeing delay bound, and a hierarchical scheme is proposed to solve the resource allocation between multiple base stations and users.

Abstract

In this paper, a two-stage intelligent scheduler is proposed to minimize the packet-level delay jitter while guaranteeing delay bound. Firstly, Lyapunov technology is employed to transform the delay-violation constraint into a sequential slot-level queue stability problem. Secondly, a hierarchical scheme is proposed to solve the resource allocation between multiple base stations and users, where the multi-agent reinforcement learning (MARL) gives the user priority and the number of scheduled packets, while the underlying scheduler allocates the resource. Our proposed scheme achieves lower delay jitter and delay violation rate than the Round-Robin Earliest Deadline First algorithm and MARL with delay violation penalty.

Figures (4)

• Figure 1: The process of the proposed intelligent packet scheduling algorithm.
• Figure 2: The comparison of delay violation ratio between LGPS-IPS and baselines;
• Figure 3: The comparison of jitter between LGPS-IPS and baselines;
• Figure 4: The delay violation ratio of different users with different $G_u$