Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A mechanism-driven reinforcement learning framework for shape optimization of airfoils

Jingfeng Wang, Guanghui Hu

TL;DR

A novel mechanism-driven reinforcement learning framework is proposed for airfoil shape optimization, which can handle the optimization problem with hundreds of design variables and is validated theoretically.

Abstract

In this paper, a novel mechanism-driven reinforcement learning framework is proposed for airfoil shape optimization. To validate the framework, a reward function is designed and analyzed, from which the equivalence between the maximizing the cumulative reward and achieving the optimization objectives is guaranteed theoretically. To establish a quality exploration, and to obtain an accurate reward from the environment, an efficient solver for steady Euler equations is employed in the reinforcement learning method. The solver utilizes the Bézier curve to describe the shape of the airfoil, and a Newton-geometric multigrid method for the solution. In particular, a dual-weighted residual-based h-adaptive method is used for efficient calculation of target functional. To effectively streamline the airfoil shape during the deformation process, we introduce the Laplacian smoothing, and propose a Bézier fitting strategy, which not only remits mesh tangling but also guarantees a precise manipulation of the geometry. In addition, a neural network architecture is designed based on an attention mechanism to make the learning process more sensitive to the minor change of the airfoil geometry. Numerical experiments demonstrate that our framework can handle the optimization problem with hundreds of design variables. It is worth mentioning that, prior to this work, there are limited works combining such high-fidelity partial differential equatons framework with advanced reinforcement learning algorithms for design problems with such high dimensionality.

A mechanism-driven reinforcement learning framework for shape optimization of airfoils

TL;DR

A novel mechanism-driven reinforcement learning framework is proposed for airfoil shape optimization, which can handle the optimization problem with hundreds of design variables and is validated theoretically.

Abstract

In this paper, a novel mechanism-driven reinforcement learning framework is proposed for airfoil shape optimization. To validate the framework, a reward function is designed and analyzed, from which the equivalence between the maximizing the cumulative reward and achieving the optimization objectives is guaranteed theoretically. To establish a quality exploration, and to obtain an accurate reward from the environment, an efficient solver for steady Euler equations is employed in the reinforcement learning method. The solver utilizes the Bézier curve to describe the shape of the airfoil, and a Newton-geometric multigrid method for the solution. In particular, a dual-weighted residual-based h-adaptive method is used for efficient calculation of target functional. To effectively streamline the airfoil shape during the deformation process, we introduce the Laplacian smoothing, and propose a Bézier fitting strategy, which not only remits mesh tangling but also guarantees a precise manipulation of the geometry. In addition, a neural network architecture is designed based on an attention mechanism to make the learning process more sensitive to the minor change of the airfoil geometry. Numerical experiments demonstrate that our framework can handle the optimization problem with hundreds of design variables. It is worth mentioning that, prior to this work, there are limited works combining such high-fidelity partial differential equatons framework with advanced reinforcement learning algorithms for design problems with such high dimensionality.
Paper Structure (17 sections, 2 theorems, 27 equations, 13 figures, 2 algorithms)

This paper contains 17 sections, 2 theorems, 27 equations, 13 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. PDEs-constrained optimization
  3. Steady Euler equations
  4. An automatic DWR-based adaptation for the target functional calculation
  5. A reinforcement learning framework for shape optimization of airfoils
  6. Markov decision process
  7. An actor-critic scheme
  8. Action of mesh deformation on unstructured mesh
  9. Design of Twin Delayed DDPG algorithm
  10. Actor and critic neural networks
  11. Exploration and exploitation
  12. Reward function
  13. Twin delayed DDPG algorithm
  14. Numerical Result
  15. Drag minimization
  16. ...and 2 more sections

Key Result

Theorem 1

Suppose that $D(\{sp^*_i\}_{i=0}^{L+1})$ represents the global minimum drag value. If the reinforcement learning process aims to maximize the cumulative reward $Q_t=\sum_{s=t}^{+\infty} R_{s}$ for each step $t$ and generate a sample points sequence with the limit $\{sp^{**}_i\}_{i=0}^{L+1}$, then th

Figures (13)

  • Figure 1: Mesh around the NACA0012 airfoil. Left: the residual-based mesh adaptation. Right: the DWR-based mesh adaptation.
  • Figure 2: Update action with a smooth function acted on the geometry with $132$ sample points.
  • Figure 3: Mesh deformation, modification, and calculation process during the reinforcement learning
  • Figure 4: Meshes around the airfoils. Left: before modifying; Right: after modifying;
  • Figure 5: Mesh around the tail. Left: before curvature capture; Right: after curvature capture.
  • ...and 8 more figures

Theorems & Definitions (4)

  • Theorem 1: Equivalence of reward function
  • proof
  • Theorem 2: Equivalence of generalized reward function
  • proof