Deep Reinforcement Learning for Network Energy Saving in 6G and Beyond Networks

TL;DR

This paper targets energy efficiency in 6G-and-beyond cellular networks by enabling selective base-station deactivation and joint optimization of antenna tilt and transmit power under heterogeneous MU QoS demands. It casts the problem as a mixed-integer nonlinear program and resolves it with a Deep Q-Network that operates on an MDP representing antenna and power control under dynamic MU locations and channel conditions. The proposed approach jointly balances time-domain (station on/off), power-domain, and space-domain (tilt) decisions to maximize the MU sum throughput while meeting RSRP and per-MU rate constraints, achieving substantial gains over heuristic baselines in large-scale simulations. The results demonstrate robust performance across varying MU densities and user distances, highlighting potential gains in 6G energy efficiency without compromising user experience.

Abstract

Network energy saving has received great attention from operators and vendors to reduce energy consumption and CO2 emissions to the environment as well as significantly reduce costs for mobile network operators. However, the design of energy-saving networks also needs to ensure the mobile users' (MUs) QoS requirements such as throughput requirements (TR). This work considers a mobile cellular network including many ground base stations (GBSs), and some GBSs are intentionally turned off due to network energy saving (NES) or crash, so the MUs located in these outage GBSs are not served in time. Based on this observation, we propose the problem of maximizing the total achievable throughput in the network by optimizing the GBSs' antenna tilt and adaptive transmission power with a given number of served MUs satisfied. Notice that, the MU is considered successfully served if its Reference Signal Received Power (RSRP) and throughput requirement are satisfied. The formulated optimization problem becomes difficult to solve with multiple binary variables and non-convex constraints along with random throughput requirements and random placement of MUs. We propose a Deep Q-learning-based algorithm to help the network learn the uncertainty and dynamics of the transmission environment. Extensive simulation results show that our proposed algorithm achieves much better performance than the benchmark schemes.

Figures (4)

• Figure 1: System model.
• Figure 2: Average rewards vs. number of iterations.
• Figure 3: Average reward per MU vs. number of MUs.
• Figure 4: Average reward vs. $d_{uk}^{\rm 3D}$.