Semi-Contention-Free Access in IoT NOMA Networks: A Reinforcement Learning Framework

TL;DR

The paper tackles scalable uplink access for massive IoT in clustered NOMA networks by introducing RL4SCF, a model-free reinforcement-learning framework that jointly optimizes access probabilities and SCF seed generation at the base station. It combines two random-access paradigms (CB and SCF) with a hash-based slot selection mechanism to reduce collisions, while devices remain lightweight, executing only a hash-based mapping. The core methodological contribution is a policy-gradient learning algorithm at the BS that updates per-cluster access policies and SCF seeds to maximize system throughput or fairness, with two reward functions guiding system-wide or fairness-oriented objectives. Across extensive simulations, RL4SCF demonstrates quasi-optimal throughput, improved fairness, reduced access delay, and lower energy consumption, showcasing its scalability to larger network sizes and traffic loads in IoT settings.

Abstract

The unprecedented surge of massive Internet of things (mIoT) traffic in beyond fifth generation (B5G) communication systems calls for transformative approaches for multiple access and data transmission. While classical model-based tools have been proven to be powerful and precise, an imminent trend for resource management in B5G networks is promoting solutions towards data-driven design. Considering an IoT network with devices spread in clusters covered by a base station, we present in this paper a novel model-free multiple access and data transmission framework empowered by reinforcement learning, designed for power-domain non-orthogonal multiple access networks to facilitate uplink traffic of small data packets. The framework supports two access modes referred to as contention-based and semi-contention-free, with its core component being a policy gradient algorithm executed at the base station. The base station performs access control and optimal radio resource allocation by periodically broadcasting two control parameters to each cluster of devices that considerably reduce data detection failures with a minimum computation requirement on devices. Numerical results, in terms of system and cluster throughput, throughput fairness, access delay, and energy consumption, demonstrate the efficiency and scalability of the framework as network size and traffic load vary.

Figures (9)

• Figure 1: Overview of the RL4SCF framework: Network scenario, frame structure, data transmission, and RL-enabled access control.
• Figure 2: PG-driven online learning: Access control and seed generation.
• Figure 3: Throughput in Scheme A when reward function $r^{(1)}$ is adopted.
• Figure 4: Throughput in Scheme A when reward function $r^{(2)}$ is adopted.
• Figure 5: Throughput in Scheme B when using reward function $r^{(1)}$.