Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Deep Reinforcement Learning Approach for Autonomous Reconfigurable Intelligent Surfaces

Hyuckjin Choi, Ly V. Nguyen, Junil Choi, A. Lee Swindlehurst

TL;DR

This work proposes the operation of an entirely autonomous RIS that operates without a control link between the RIS and a base station, and employs a deep Q network (DQN) based on reinforcement learning in order to enhance the sum rate of the network.

Abstract

A reconfigurable intelligent surface (RIS) is a prospective wireless technology that enhances wireless channel quality. An RIS is often equipped with passive array of elements and provides cost and power-efficient solutions for coverage extension of wireless communication systems. Without any radio frequency (RF) chains or computing resources, however, the RIS requires control information to be sent to it from an external unit, e.g., a base station (BS). The control information can be delivered by wired or wireless channels, and the BS must be aware of the RIS and the RIS-related channel conditions in order to effectively configure its behavior. Recent works have introduced hybrid RIS structures possessing a few active elements that can sense and digitally process received data. Here, we propose the operation of an entirely autonomous RIS that operates without a control link between the RIS and BS. Using a few sensing elements, the autonomous RIS employs a deep Q network (DQN) based on reinforcement learning in order to enhance the sum rate of the network. Our results illustrate the potential of deploying autonomous RISs in wireless networks with essentially no network overhead.

A Deep Reinforcement Learning Approach for Autonomous Reconfigurable Intelligent Surfaces

TL;DR

This work proposes the operation of an entirely autonomous RIS that operates without a control link between the RIS and a base station, and employs a deep Q network (DQN) based on reinforcement learning in order to enhance the sum rate of the network.

Abstract

A reconfigurable intelligent surface (RIS) is a prospective wireless technology that enhances wireless channel quality. An RIS is often equipped with passive array of elements and provides cost and power-efficient solutions for coverage extension of wireless communication systems. Without any radio frequency (RF) chains or computing resources, however, the RIS requires control information to be sent to it from an external unit, e.g., a base station (BS). The control information can be delivered by wired or wireless channels, and the BS must be aware of the RIS and the RIS-related channel conditions in order to effectively configure its behavior. Recent works have introduced hybrid RIS structures possessing a few active elements that can sense and digitally process received data. Here, we propose the operation of an entirely autonomous RIS that operates without a control link between the RIS and BS. Using a few sensing elements, the autonomous RIS employs a deep Q network (DQN) based on reinforcement learning in order to enhance the sum rate of the network. Our results illustrate the potential of deploying autonomous RISs in wireless networks with essentially no network overhead.
Paper Structure (18 sections, 28 equations, 6 figures)

This paper contains 18 sections, 28 equations, 6 figures.

Table of Contents

  1. Introduction
  2. Channel Model
  3. System Model
  4. Proposed Sum-Rate Evaluation
  5. Observation Recovery
  6. Sum-Rate Evaluation Method
  7. Proposed DQN Design
  8. DQN Arguments
  9. Environment
  10. Agent
  11. State
  12. Action
  13. Policy
  14. Reward
  15. DQN structure
  16. ...and 3 more sections

Figures (6)

  • Figure 1: Illustration of an autonomous RIS-assisted system.
  • Figure 2: Assumed cluster-based channel model. The direct links between BS and UEs are blocked.
  • Figure 3: Flowchart of the proposed DQN interacting with the environment.
  • Figure 4: Structure of the DQN neural network with two input pipelines. The number of weight parameters is written on each layer.
  • Figure 5: The procedure associated with the proposed DQN for autonomous RIS. The training sample period is $N_c/2W_s$, where $W_s$ is baseband bandwidth, and $N_c$ is the sampling interval.
  • ...and 1 more figures