Optimal Power Allocation and Sub-Optimal Channel Assignment for Downlink NOMA Systems Using Deep Reinforcement Learning

TL;DR

The paper addresses efficient resource allocation in downlink NOMA by combining a mathematically optimal power allocation (JRA) with a DRL-based channel assignment. It introduces a policy-gradient DRL framework that incorporates replay memory to improve generalization across dynamic environments, training with multiple architectures and hyperparameters. Evaluations show that JRA-DRL yields sum-rate performance close to exhaustive search while offering substantially faster channel allocation compared to ES, and outperforms JRA alone due to improved policy generalization. This approach provides a scalable, generalizable solution for real-time resource management in IoT-enabled networks with NOMA and SIC.

Abstract

In recent years, Non-Orthogonal Multiple Access (NOMA) system has emerged as a promising candidate for multiple access frameworks due to the evolution of deep machine learning, trying to incorporate deep machine learning into the NOMA system. The main motivation for such active studies is the growing need to optimize the utilization of network resources as the expansion of the internet of things (IoT) caused a scarcity of network resources. The NOMA addresses this need by power multiplexing, allowing multiple users to access the network simultaneously. Nevertheless, the NOMA system has few limitations. Several works have proposed to mitigate this, including the optimization of power allocation known as joint resource allocation(JRA) method, and integration of the JRA method and deep reinforcement learning (JRA-DRL). Despite this, the channel assignment problem remains unclear and requires further investigation. In this paper, we propose a deep reinforcement learning framework incorporating replay memory with an on-policy algorithm, allocating network resources in a NOMA system to generalize the learning. Also, we provide extensive simulations to evaluate the effects of varying the learning rate, batch size, type of model, and the number of features in the state.

Figures (9)

• Figure 1: Block diagram illustrating the transmission of BS and reception of users of the downlink NOMA system.
• Figure 2: Sum rate comparison of different learning rates with $N \times K \times F = 6 \times 3 \times 3$, batch size = $40$ model = FCNN and $P_{T} = 12 W$.
• Figure 3: Sum rate comparison of different batch sizes with $N \times K \times F = 6 \times 3 \times 3$, learning rate = $0.005$, model = FCNN and $P_{T} = 12 W$.
• Figure 4: Loss comparison of different number features with $N \times K = 6 \times 3$, learning rate = $0.005$, batch size = $40$, model = FCNN and $P_{T} = 12 W$.
• Figure 5: Loss comparison of different models with $N \times K \times F = 6 \times 3 \times 3$, learning rate = $0.005$, batch size = $40$ and $P_{T} = 12 W$.