Reinforcement learning for anisotropic p-adaptation and error estimation in high-order solvers

TL;DR

This work addresses automated, anisotropic $p$-adaptation in high-order DG solvers by introducing an offline-trained reinforcement learning agent based on value iteration. The agent operates on discretized per-element states derived from Gauss-node values, enabling mesh-agnostic deployment and anisotropic refinement across 3D problems, while an inexpensive RL-based error estimator quantifies local discretization error. The methodology is implemented in HORSES3D and validated on a progression from Euler cylinder flow to 3D turbulence in a wind turbine simulation, demonstrating accurate solutions with substantial reductions in computational cost and without retraining for new cases. The results indicate strong potential for generalizing high-order, RL-driven mesh adaptation to complex PDEs beyond CFD, with robust error tracking and practical offline training benefits.

Abstract

We present a novel approach to automate and optimize anisotropic p-adaptation in high-order h/p solvers using Reinforcement Learning (RL). The dynamic RL adaptation uses the evolving solution to adjust the high-order polynomials. We develop an offline training approach, decoupled from the main solver, which shows minimal overcost when performing simulations. In addition, we derive an inexpensive RL-based error estimation approach that enables the quantification of local discretization errors. The proposed methodology is agnostic to both the computational mesh and the partial differential equation to be solved. The application of RL to mesh adaptation offers several benefits. It enables automated and adaptive mesh refinement, reducing the need for manual intervention. It optimizes computational resources by dynamically allocating high-order polynomials where necessary and minimizing refinement in stable regions. This leads to computational cost savings while maintaining the accuracy of the solution. Furthermore, RL allows for the exploration of unconventional mesh adaptations, potentially enhancing the accuracy and robustness of simulations. This work extends our original research, offering a more robust, reproducible, and generalizable approach applicable to complex three-dimensional problems. We provide validation for laminar and turbulent cases: circular cylinders, Taylor Green Vortex and a 10MW wind turbine to illustrate the flexibility of the proposed approach.

Figures (18)

• Figure 1: Geometrical transformations in the DGSEM method thesis_Juan_DG.
• Figure 2: Example of a 1D subdivision into finite elements with piece-wise solutions.
• Figure 3: Schematic definition of the RL framework.
• Figure 4: Example of a comparison between the approximate solution $y^*$ ($p=3$ in this example) and the low-order reference solution ($p=2$ in this example). The error between both functions is estimated through the root mean squared error defined in eq. (\ref{['eq:rmse_reward']}).
• Figure 5: a) Values of the state vector for an approximate solution $y^*(x)$ and b) reference solutions $y(x)$ calculated during the training process.