Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Machine Learning-enabled Traffic Steering in O-RAN: A Case Study on Hierarchical Learning Approach

Md Arafat Habib, Hao Zhou, Pedro Enrique Iturria-Rivera, Yigit Ozcan, Medhat Elsayed, Majid Bavand, Raimundas Gaigalas, Melike Erol-Kantarci

TL;DR

A hierarchical deep-Q-learning (h-DQN) framework for traffic steering is proposed, which decomposes the traffic steering problem into a bi-level architecture with hierarchical intelligence.

Abstract

Traffic Steering is a crucial technology for wireless networks, and multiple efforts have been put into developing efficient Machine Learning (ML)-enabled traffic steering schemes for Open Radio Access Networks (O-RAN). Given the swift emergence of novel ML techniques, conducting a timely survey that comprehensively examines the ML-based traffic steering schemes in O-RAN is critical. In this article, we provide such a survey along with a case study of hierarchical learning-enabled traffic steering in O-RAN. In particular, we first introduce the background of traffic steering in O-RAN and overview relevant state-of-the-art ML techniques and their applications. Then, we analyze the compatibility of the hierarchical learning framework in O-RAN and further propose a Hierarchical Deep-Q-Learning (h-DQN) framework for traffic steering. Compared to existing works, which focus on single-layer architecture with standalone agents, h-DQN decomposes the traffic steering problem into a bi-level architecture with hierarchical intelligence. The meta-controller makes long-term and high-level policies, while the controller executes instant traffic steering actions under high-level policies. Finally, the case study shows that the hierarchical learning approach can provide significant performance improvements over the baseline algorithms.

Machine Learning-enabled Traffic Steering in O-RAN: A Case Study on Hierarchical Learning Approach

TL;DR

A hierarchical deep-Q-learning (h-DQN) framework for traffic steering is proposed, which decomposes the traffic steering problem into a bi-level architecture with hierarchical intelligence.

Abstract

Traffic Steering is a crucial technology for wireless networks, and multiple efforts have been put into developing efficient Machine Learning (ML)-enabled traffic steering schemes for Open Radio Access Networks (O-RAN). Given the swift emergence of novel ML techniques, conducting a timely survey that comprehensively examines the ML-based traffic steering schemes in O-RAN is critical. In this article, we provide such a survey along with a case study of hierarchical learning-enabled traffic steering in O-RAN. In particular, we first introduce the background of traffic steering in O-RAN and overview relevant state-of-the-art ML techniques and their applications. Then, we analyze the compatibility of the hierarchical learning framework in O-RAN and further propose a Hierarchical Deep-Q-Learning (h-DQN) framework for traffic steering. Compared to existing works, which focus on single-layer architecture with standalone agents, h-DQN decomposes the traffic steering problem into a bi-level architecture with hierarchical intelligence. The meta-controller makes long-term and high-level policies, while the controller executes instant traffic steering actions under high-level policies. Finally, the case study shows that the hierarchical learning approach can provide significant performance improvements over the baseline algorithms.
Paper Structure (15 sections, 5 figures, 1 table)

This paper contains 15 sections, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Traffic Steering in O-RAN
  3. ML-enabled Traffic Steering Schemes
  4. State-of-the-art ML Approaches for Traffic Steering
  5. Deep Learning
  6. Reinforcement Learning
  7. Federated Learning
  8. Hierarchical Learning
  9. Potential ML paradigms for Traffic Steering
  10. Hierarchical Learning Based Traffic Steering
  11. Hierarchical Learning Scheme
  12. Hierarchical Deep-Q-Network based Traffic Steering Apps
  13. OTIC Integration and O-RAN AI/ML Workflow:
  14. A Case Study with Hierarchical Reinforcement Learning for 5G
  15. Conclusions

Figures (5)

  • Figure 1: O-RAN architecture with traffic steering xApp.
  • Figure 2: Hierarchical learning scheme for O-RAN.
  • Figure 3: Hierarchical Reinforcement Learning Architecture for Traffic Steering: hierarchical deep Q-network.
  • Figure 4: Performance achievement of hierarchical reinforcement learning versus baselines in terms of throughput and delay.
  • Figure 5: Traffic being steered to a different RAT based on load balancing threshold.