Knowledge Defined Networking for 6G: A Reinforcement Learning Example for Resource Management
Erol Koçoğlu, Mehmet Ozdem, Tuğçe Bilen
TL;DR
The paper tackles the challenge of resource management in 6G networks by integrating Knowledge-Defined Networking (KDN) with reinforcement learning. It proposes a centralized RL agent within a KDN framework that uses telemetry, SDN, and a knowledge plane to optimize UE association and base-station power control via discrete actions. The RL model employs Q-learning with an $\epsilon$-greedy policy and a reward structure based on latency, packet loss, and throughput, formalized through the update $Q(s,a) \leftarrow Q(s,a) + \alpha [ r + \gamma \max_{a'} Q(s',a') - Q(s,a) ]$ and state $s$ consisting of metrics like $\text{Packet Loss}$, $\text{Latency}$, $\text{Throughput}$, $\text{UE Mobility}$, and $\text{Load Distribution}$. Simulation studies using ns-3/ns3-gym with the TerasSim THz extension demonstrate that RL-KDN outperforms a baseline idle policy in throughput, latency, and packet loss, especially under high user densities, validating the approach's practicality for ultra-dense 6G environments.
Abstract
6G networks are expected to revolutionize connectivity, offering significant improvements in speed, capacity, and smart automation. However, existing network designs will struggle to handle the demands of 6G, which include much faster speeds, a huge increase in connected devices, lower energy consumption, extremely quick response times, and better mobile broadband. To solve this problem, incorporating the artificial intelligence (AI) technologies has been proposed. This idea led to the concept of Knowledge-Defined Networking (KDN). KDN promises many improvements, such as resource management, routing, scheduling, clustering, and mobility prediction. The main goal of this study is to optimize resource management using Reinforcement Learning.