Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Design and Evaluation of Deep Reinforcement Learning for Energy Saving in Open RAN

Matteo Bordin, Andrea Lacava, Michele Polese, Sai Satish, Manoj AnanthaSwamy Nittoor, Rajarajan Sivaraj, Francesca Cuomo, Tommaso Melodia

TL;DR

Results show that DRL agents improve energy efficiency by adapting to network conditions while minimally impacting the user experience, and the trade-off between throughput and energy consumption offered by different DRL agent designs.

Abstract

Next-generation wireless systems, already widely deployed, are expected to become even more prevalent in the future, representing challenges in both environmental and economic terms. This paper focuses on improving the energy efficiency of intelligent and programmable Open Radio Access Network (RAN) systems through the near-real-time dynamic activation and deactivation of Base Station (BS) Radio Frequency (RF) frontends using Deep Reinforcement Learning (DRL) algorithms, i.e., Proximal Policy Optimization (PPO) and Deep Q-Network (DQN). These algorithms run on the RAN Intelligent Controllers (RICs), part of the Open RAN architecture, and are designed to make optimal network-level decisions based on historical data without compromising stability and performance. We leverage a rich set of Key Performance Measurements (KPMs), serving as state for the DRL, to create a comprehensive representation of the RAN, alongside a set of actions that correspond to some control exercised on the RF frontend. We extend ns-O-RAN, an open-source, realistic simulator for 5G and Open RAN built on ns-3, to conduct an extensive data collection campaign. This enables us to train the agents offline with over 300,000 data points and subsequently evaluate the performance of the trained models. Results show that DRL agents improve energy efficiency by adapting to network conditions while minimally impacting the user experience. Additionally, we explore the trade-off between throughput and energy consumption offered by different DRL agent designs.

Design and Evaluation of Deep Reinforcement Learning for Energy Saving in Open RAN

TL;DR

Results show that DRL agents improve energy efficiency by adapting to network conditions while minimally impacting the user experience, and the trade-off between throughput and energy consumption offered by different DRL agent designs.

Abstract

Next-generation wireless systems, already widely deployed, are expected to become even more prevalent in the future, representing challenges in both environmental and economic terms. This paper focuses on improving the energy efficiency of intelligent and programmable Open Radio Access Network (RAN) systems through the near-real-time dynamic activation and deactivation of Base Station (BS) Radio Frequency (RF) frontends using Deep Reinforcement Learning (DRL) algorithms, i.e., Proximal Policy Optimization (PPO) and Deep Q-Network (DQN). These algorithms run on the RAN Intelligent Controllers (RICs), part of the Open RAN architecture, and are designed to make optimal network-level decisions based on historical data without compromising stability and performance. We leverage a rich set of Key Performance Measurements (KPMs), serving as state for the DRL, to create a comprehensive representation of the RAN, alongside a set of actions that correspond to some control exercised on the RF frontend. We extend ns-O-RAN, an open-source, realistic simulator for 5G and Open RAN built on ns-3, to conduct an extensive data collection campaign. This enables us to train the agents offline with over 300,000 data points and subsequently evaluate the performance of the trained models. Results show that DRL agents improve energy efficiency by adapting to network conditions while minimally impacting the user experience. Additionally, we explore the trade-off between throughput and energy consumption offered by different DRL agent designs.

Paper Structure

This paper contains 10 sections, 2 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. State of the Art
  3. System overview
  4. DRL Agent Design
  5. Simulation Setup
  6. Simulation Scenario
  7. Energy-Saving Features and Baselines in ns-O-RAN
  8. Training Dataset
  9. Results
  10. Conclusions

Figures (4)

  • Figure 1: System architecture
  • Figure 2: Simulation scenario overview.
  • Figure 3: Comparison across multiple for the baselines, PPO, and DQN agents.
  • Figure 4: Percentage of the average Throughput compared to the percentage of the average Energy Consumption.