Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Heuristic-Integrated DRL Approach for Phase Optimization in Large-Scale RISs

Wei Wang, Peizheng Li, Angela Doufexi, Mark A. Beach

TL;DR

This work tackles the challenging discrete phase-shift optimization in large-scale RISs by introducing a heuristic-integrated DRL framework. The method uses accumulated column-wise actions within a double-DQN and couples a greedy algorithm at each step for fine-grained element-wise refinement, effectively reducing the action space from $2^{N R_{ heta}}$ toward $O( ext{sqrt}(N))$ while preserving performance. The DDQN-GA approach achieves higher sum-rate than conventional DRL baselines and scales to larger RIS sizes, thanks to the combination of reduced action complexity and targeted element-wise optimization. This has practical significance for real-time RIS optimization in dense deployments, enabling near-optimal configurations with manageable training and inference overhead.

Abstract

Optimizing discrete phase shifts in large-scale reconfigurable intelligent surfaces (RISs) is challenging due to their non-convex and non-linear nature. In this letter, we propose a heuristic-integrated deep reinforcement learning (DRL) framework that (1) leverages accumulated actions over multiple steps in the double deep Q-network (DDQN) for RIS column-wise control and (2) integrates a greedy algorithm (GA) into each DRL step to refine the state via fine-grained, element-wise optimization of RIS configurations. By learning from GA-included states, the proposed approach effectively addresses RIS optimization within a small DRL action space, demonstrating its capability to optimize phase-shift configurations of large-scale RISs.

A Heuristic-Integrated DRL Approach for Phase Optimization in Large-Scale RISs

TL;DR

This work tackles the challenging discrete phase-shift optimization in large-scale RISs by introducing a heuristic-integrated DRL framework. The method uses accumulated column-wise actions within a double-DQN and couples a greedy algorithm at each step for fine-grained element-wise refinement, effectively reducing the action space from toward while preserving performance. The DDQN-GA approach achieves higher sum-rate than conventional DRL baselines and scales to larger RIS sizes, thanks to the combination of reduced action complexity and targeted element-wise optimization. This has practical significance for real-time RIS optimization in dense deployments, enabling near-optimal configurations with manageable training and inference overhead.

Abstract

Optimizing discrete phase shifts in large-scale reconfigurable intelligent surfaces (RISs) is challenging due to their non-convex and non-linear nature. In this letter, we propose a heuristic-integrated deep reinforcement learning (DRL) framework that (1) leverages accumulated actions over multiple steps in the double deep Q-network (DDQN) for RIS column-wise control and (2) integrates a greedy algorithm (GA) into each DRL step to refine the state via fine-grained, element-wise optimization of RIS configurations. By learning from GA-included states, the proposed approach effectively addresses RIS optimization within a small DRL action space, demonstrating its capability to optimize phase-shift configurations of large-scale RISs.
Paper Structure (8 sections, 8 equations, 3 figures, 1 table, 2 algorithms)

This paper contains 8 sections, 8 equations, 3 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. System Model
  3. Proposed Heuristic-integrated DRL Optimization Algorithm
  4. Proposed DRL Algorithm for Column-wise Phase Shift Design
  5. Integration of Heuristic Algorithm for Element-wise Phase Shift Design
  6. Complexity Analysis
  7. Numerical Results
  8. Conclusion

Figures (3)

  • Figure 1: Illustration of the proposed DDQN and DDQN-Greedy schemes.
  • Figure 2: Action space size for discrete DRL algorithms with different control schemes and RIS phase shift resolutions. The x-axis represents the number of RIS unit cells in one dimension of a rectangular RIS array.
  • Figure 3: Numerical results: (a) Averaged and smoothed DRL training curves under different numbers of steps per episode; (b) Instant and averaged DRL training curves for various optimization methods; (c) Sum rate versus RIS size.