A Reinforcement Learning Framework for Resource Allocation in Uplink Carrier Aggregation in the Presence of Self Interference

TL;DR

This work tackles uplink carrier aggregation resource allocation under non-linear PA-induced self-interference by formulating it as a mixed discrete-continuous optimization problem. It introduces a centralized multi-agent CA2C reinforcement learning framework with a novel SI-aware reward (soft avoidance) that jointly optimizes carrier activation and power allocation while mitigating DL sensitivity degradation. The authors show that soft avoidance with fine-grained RB allocation yields higher sum throughput than hard-avoidance or baseline methods, and that the approach remains effective under SI presence/absence and under imperfect CSI. The proposed method demonstrates online adaptability to dynamic traffic and user changes, with practical computational requirements and potential applicability to downlink CA as well. Overall, the paper provides a novel, SI-conscious RL solution for uplink CA that improves throughput and offers a pathway to scalable, real-time network optimization.

Abstract

Carrier aggregation (CA) is a technique that allows mobile networks to combine multiple carriers to increase user data rate. On the uplink, for power constrained users, this translates to the need for an efficient resource allocation scheme, where each user distributes its available power among its assigned uplink carriers. Choosing a good set of carriers and allocating appropriate power on the carriers is important. If the carrier allocation on the uplink is such that a harmonic of a user's uplink carrier falls on the downlink frequency of that user, it leads to a self coupling-induced sensitivity degradation of that user's downlink receiver. In this paper, we model the uplink carrier aggregation problem as an optimal resource allocation problem with the associated constraints of non-linearities induced self interference (SI). This involves optimization over a discrete variable (which carriers need to be turned on) and a continuous variable (what power needs to be allocated on the selected carriers) in dynamic environments, a problem which is hard to solve using traditional methods owing to the mixed nature of the optimization variables and the additional need to consider the SI constraint. We adopt a reinforcement learning (RL) framework involving a compound-action actor-critic (CA2C) algorithm for the uplink carrier aggregation problem. We propose a novel reward function that is critical for enabling the proposed CA2C algorithm to efficiently handle SI. The CA2C algorithm along with the proposed reward function learns to assign and activate suitable carriers in an online fashion. Numerical results demonstrate that the proposed RL based scheme is able to achieve higher sum throughputs compared to naive schemes. The results also demonstrate that the proposed reward function allows the CA2C algorithm to adapt the optimization both in the presence and absence of SI.

Figures (22)

• Figure 1: Network environment.
• Figure 2: SI due to PA non-linearity at UE.
• Figure 3: Receiver sensitivity degradation as a function of SI power level $(p_\text{SI})_\text{Rx,in}$.
• Figure 4: RL approach with one agent per gNB.
• Figure 5: RL architecture using actor-critic network.