Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Deep Reinforcement Learning-based Approach for Adaptive Handover Protocols in Mobile Networks

Peter J. Gu, Johannes Voigt, Peter M. Rost

TL;DR

This paper addresses adaptive handover in dense 5G NR networks where frequent handovers cause outages and ping-pong effects. It introduces a proximal policy optimization (PPO) based reinforcement learning agent that learns an adaptive handover policy using RSRP-driven state representations and models HO preparation and execution times with timers. Across speeds, the PPO agent achieves higher mean data rate, quantified by $Γ$, and fewer radio link failures than the fixed-parameter 3GPP handover protocol, within a reproducible Vienna 5G/QuaDRiGa simulation framework. The work demonstrates the potential of DRL to improve mobility management in next-generation networks and provides code for reproducibility.

Abstract

Due to an ever-increasing number of participants and new areas of application, the demands on mobile communications systems are continually increasing. In order to deliver higher data rates, enable mobility and guarantee QoS requirements of subscribers, these systems and the protocols used are becoming more complex. By using higher frequency spectrums, cells become smaller and more base stations have to be deployed. This leads to an increased number of handovers of user equipments between base stations in order to enable mobility, resulting in potentially more frequent radio link failures and rate reduction. The persistent switching between the same base stations, commonly referred to as "ping-pong", leads to a consistent reduction of data rates. In this work, we propose a method for handover optimization by using proximal policy optimization in mobile communications to learn an adaptive handover protocol. The resulting agent is highly flexible regarding different travelling speeds of user equipments, while outperforming the standard 5G NR handover protocol by 3GPP in terms of average data rate and number of radio link failures. Furthermore, the design of the proposed environment demonstrates remarkable accuracy, ensuring a fair comparison with the standard 3GPP protocol.

A Deep Reinforcement Learning-based Approach for Adaptive Handover Protocols in Mobile Networks

TL;DR

This paper addresses adaptive handover in dense 5G NR networks where frequent handovers cause outages and ping-pong effects. It introduces a proximal policy optimization (PPO) based reinforcement learning agent that learns an adaptive handover policy using RSRP-driven state representations and models HO preparation and execution times with timers. Across speeds, the PPO agent achieves higher mean data rate, quantified by , and fewer radio link failures than the fixed-parameter 3GPP handover protocol, within a reproducible Vienna 5G/QuaDRiGa simulation framework. The work demonstrates the potential of DRL to improve mobility management in next-generation networks and provides code for reproducibility.

Abstract

Due to an ever-increasing number of participants and new areas of application, the demands on mobile communications systems are continually increasing. In order to deliver higher data rates, enable mobility and guarantee QoS requirements of subscribers, these systems and the protocols used are becoming more complex. By using higher frequency spectrums, cells become smaller and more base stations have to be deployed. This leads to an increased number of handovers of user equipments between base stations in order to enable mobility, resulting in potentially more frequent radio link failures and rate reduction. The persistent switching between the same base stations, commonly referred to as "ping-pong", leads to a consistent reduction of data rates. In this work, we propose a method for handover optimization by using proximal policy optimization in mobile communications to learn an adaptive handover protocol. The resulting agent is highly flexible regarding different travelling speeds of user equipments, while outperforming the standard 5G NR handover protocol by 3GPP in terms of average data rate and number of radio link failures. Furthermore, the design of the proposed environment demonstrates remarkable accuracy, ensuring a fair comparison with the standard 3GPP protocol.
Paper Structure (19 sections, 10 equations, 5 figures, 2 tables, 2 algorithms)

This paper contains 19 sections, 10 equations, 5 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. preliminaries
  3. Handover Procedure in 5G NR
  4. Reinforcement Learning
  5. Handover Optimization Problem
  6. System Model
  7. RL Environment Design
  8. Simulation of Ping-Pongs
  9. Simulation of Handover Failures
  10. Simulation Overall
  11. Pseudo RSRQ
  12. State and Action Design
  13. Reward Design
  14. Training Process
  15. Learning Algorithm
  16. ...and 4 more sections

Figures (5)

  • Figure 1: Schematic of RL.
  • Figure 2: Flow diagram visualizing the actor-critic update process.
  • Figure 3: Simulation area of Karlsruhe with 5 BS. The black circles represent the BS, while the lines indicate the direction of the beam.
  • Figure 4: HOF triggered after HO preparation.
  • Figure 5: Simulation results for the 3GPP protocol and the PPO agent for one evaluation data set.