Multi-User Multi-Application Packet Scheduling for Application-Specific QoE Enhancement Based on Knowledge-Embedded DDPG in 6G RAN

TL;DR

The paper tackles multi-user multi-application packet scheduling in downlink 6G RAN to maximize inter-UE fairness of application-specific QoE, formulating the problem as an $MDP$ and solving it with a $DDPG$ agent augmented by a knowledge embedding mechanism. By modelling per-application QoE with parametric functions and using CQI/MCS mappings, buffers, and QoS constraints, the approach targets tailored QoE while maintaining fairness. The authors demonstrate that the $DDPG$ scheduler outperforms baseline schedulers, and that knowledge embedding further improves performance across diverse applications (FTP, UHD, Web, Online Gaming, VoIP). These results highlight the method’s potential to enable fine-grained QoE provisioning and fair resource allocation in future 6G networks, with practical impact for diverse services and UEs.

Abstract

The rapidly growing diversity of concurrent applications from both different users and same devices calls for application-specific Quality of Experience (QoE) enhancement of future wireless communications. Achieving this goal relies on application-specific packet scheduling, as it is vital for achieving tailored QoE enhancement by realizing the application-specific Quality of Service (QoS) requirements and for optimal perceived QoE values. However, the intertwining diversified QoE perception mechanisms, fairness among concurrent applications, and the impact of network dynamics inevitably complicate tailored packet scheduling. To achieve concurrent application-specific QoE enhancement, the problem of multi-user multi-application packet scheduling in downlink 6G radio access network (RAN) is first formulated as a Markov decision process (MDP) problem in this paper. For solving this problem, a deep deterministic policy gradient (DDPG)-based solution is proposed. However, due to the high dimensionalities of both the state and action spaces, the trained DDPG agents might generate decisions causing unnecessary resource waste. Hence, a knowledge embedding method is proposed to adjust the decisions of the DDPG agents according to human insights. Extensive experiments are conducted, which demonstrate the superiority of DDPG-based packet schedulers over baseline algorithms and the effectiveness of the proposed knowledge embedding technique.

Figures (7)

• Figure 1: Illustration of the process of multi-user multi-application packet scheduling in downlink 6G RAN. The packet scheduler takes information from both physical and MAC layers, as well as the QoS requirements and packet delay budgets as inputs to generate packet scheduling policy. The CQI values of UEs are first transformed into modulation schemes and code rates by CQI / MCS mapper, then forwarded to packet scheduler as inputs.
• Figure 2: Illustration of the design of actor network of DDPG algorithm. The actor network is composed of one input layer, one flatten layer, a number of fully connected layers activated by ReLU function as middle layers (the number of middle layers is set to be 3 in this figure) , and one fully connected layer activated by ReLU function as output layer.
• Figure 3: Illustration of the design of critic network of DDPG algorithm. The critic network contains two branches as it takes two separate inputs. The first branch is composed of one input layer, one flatter layer and a number of fully connected layers activated by ReLU function as middle layers (the number of middle layers is set to be 2 in this figure). The second branch is composed of one input layer and one fully connected layer activated by ReLU function. The two branches are connected to one concatenate layer followed by two fully connected layers activated by ReLU function to generate the output value of the input (state, action) pair.
• Figure 4: Simulation results of DDPG algorithm under the four settings during the training stage.
• Figure 5: Performance comparison of DDPG and KE-DDPG-based packet schedulers under the four settings on the test dataset. Under each setting, under the majority of testing episodes, KE-DDPG-based packet scheduler outperforms DDPG-based packet scheduler, indicating that knowledge embedding is effective to improve the performance of DDPG-based packet schedulers.

Theorems & Definitions (3)

• Definition 1
• Definition 2
• Definition 3