Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Soft Actor-Critic with Backstepping-Pretrained DeepONet for control of PDEs

Chenchen Wang, Jie Qi, Jiaqi Hu

Abstract

This paper develops a reinforcement learning-based controller for the stabilization of partial differential equation (PDE) systems. Within the soft actor-critic (SAC) framework, we embed a DeepONet, a well-known neural operator (NO), which is pretrained using the backstepping controller. The pretrained DeepONet captures the essential features of the backstepping controller and serves as a feature extractor, replacing the convolutional neural networks (CNNs) layers in the original actor and critic networks, and directly connects to the fully connected layers of the SAC architecture. We apply this novel backstepping and reinforcement learning integrated method to stabilize an unstable ffrst-order hyperbolic PDE and an unstable reactiondiffusion PDE. Simulation results demonstrate that the proposed method outperforms the standard SAC, SAC with an untrained DeepONet, and the backstepping controller on both systems.

Soft Actor-Critic with Backstepping-Pretrained DeepONet for control of PDEs

Abstract

This paper develops a reinforcement learning-based controller for the stabilization of partial differential equation (PDE) systems. Within the soft actor-critic (SAC) framework, we embed a DeepONet, a well-known neural operator (NO), which is pretrained using the backstepping controller. The pretrained DeepONet captures the essential features of the backstepping controller and serves as a feature extractor, replacing the convolutional neural networks (CNNs) layers in the original actor and critic networks, and directly connects to the fully connected layers of the SAC architecture. We apply this novel backstepping and reinforcement learning integrated method to stabilize an unstable ffrst-order hyperbolic PDE and an unstable reactiondiffusion PDE. Simulation results demonstrate that the proposed method outperforms the standard SAC, SAC with an untrained DeepONet, and the backstepping controller on both systems.

Paper Structure

This paper contains 11 sections, 11 equations, 3 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem description
  3. General PDE control problem
  4. Backstepping control operator
  5. DeepONet for learning backstepping controller
  6. NO-based reinforcement learning framework
  7. Actor network
  8. Critic networks
  9. Algorithm (Markov Decision Process)
  10. Simulation
  11. Simulation of a 1D Hyperbolic PDE

Figures (3)

  • Figure 1: Architecture of DeepONet as feature extractor for SAC.
  • Figure 2: The DeepONet pre-trained with the backstepping method is embedded into the SAC framework.
  • Figure :