Federated Reinforcement Learning for Uplink Centric Broadband Communication Optimization over Unlicensed Spectrum

TL;DR

The paper tackles uplink-centric broadband optimization over unlicensed spectrum where NR-U must coexist with WiFi. It introduces a dual DRL approach, first a centralized double-DQN (DDQN) and then a privacy-preserving federated DDQN, to adapt energy-detection thresholds $( ext{λ}^{ED}_w, ext{λ}^{ED}_n)$ for NR-U and WiFi, respectively; learning is guided by a newly proposed user-centric throughput metric (UPT) and a fairness-aware reward function. Key contributions include formulating the problem as a POMDP, detailing centralized and federated learning workflows with DDQN and FedAvg, and showing substantial uplink throughput gains (often exceeding $100">$ in system throughput and over $100 ext%$ uplink gains) while maintaining or improving WiFi performance under fairness constraints. The results highlight the potential of DRL-based ED-threshold tuning to enhance UCBC in mixed NR-U and WiFi environments, with practical impact for 5.5G uplink efficiency and coexistence fairness.

Abstract

To provide Uplink Centric Broadband Communication (UCBC), New Radio Unlicensed (NR-U) network has been standardized to exploit the unlicensed spectrum using Listen Before Talk (LBT) scheme to fairly coexist with the incumbent Wireless Fidelity (WiFi) network. Existing access schemes over unlicensed spectrum are required to perform Clear Channel Assessment (CCA) before transmissions, where fixed Energy Detection (ED) thresholds are adopted to identify the channel as idle or busy. However, fixed ED thresholds setting prevents devices from accessing the channel effectively and efficiently, which leads to the hidden node (HN) and exposed node (EN) problems. In this paper, we first develop a centralized double Deep Q-Network (DDQN) algorithm to optimize the uplink system throughput, where the agent is deployed at the central server to dynamically adjust the ED thresholds for NR-U and WiFi networks. Considering that heterogeneous NR-U and WiFi networks, in practice, cannot share the raw data with the central server directly, we then develop a federated DDQN algorithm, where two agents are deployed in the NR-U and WiFi networks, respectively. Our results have shown that the uplink system throughput increases by over 100%, where cell throughput of NR-U network rises by 150%, and cell throughput of WiFi network decreases by 30%. To guarantee the cell throughput of WiFi network, we redesign the reward function to punish the agent when the cell throughput of WiFi network is below the threshold, and our revised design can still provide over 50% uplink system throughput gain.

Figures (11)

• Figure 1: Uplink transmission of NR-U and WiFi indoor coexistence scenario.
• Figure 2: Timing graph of uplink transmission for WiFi and NR-U networks.
• Figure 3: Procedures of gNB scheduling and UE data transmission.
• Figure 4: Hidden node problem and exposed node problem.
• Figure 5: The DQN agent and environment interaction in the POMDP.