Reinforcement Learning-Based Filters for Convection-Dominated Flows: Reference-Free and Reference-Guided Training

TL;DR

A reinforcement learning (RL) framework for the dynamic selection of the filter parameter in Evolve-Filter regularization strategies for incompressible turbulent flows provides a robust and flexible alternative to manually tuned regularization parameters, enabling adaptive, physically consistent control of filtering in turbulent flow simulations.

Abstract

We propose a reinforcement learning (RL) framework for the dynamic selection of the filter parameter in Evolve-Filter (EF) regularization strategies for incompressible turbulent flows. Instead of prescribing the filter radius heuristically, the RL agent learns to adaptively control the filtering intensity in time, balancing numerical stability and physical accuracy. The methodology is assessed on two benchmark problems with fundamentally different dynamics: flow past a cylinder and decaying homogeneous turbulence. Both reference-guided and reference-free reward formulations are investigated. In the reference-guided setting, the agent is trained using direct numerical simulation (DNS) data over a limited time window and then evaluated in extrapolation. In the reference-free setting, the reward relies exclusively on physics-based quantities, without access to reference solutions, i.e., completely eliminating the computational costs related to DNS simulations. The results show that the proposed RL-EF strategies prevent numerical blow-up while avoiding the excessive dissipation typical of standard EF approaches based on a fixed Kolmogorov length scale. The learned policies accurately reproduce the relevant flow dynamics across scales, preserving the correct balance between large-scale structures and small-scale dissipation. Notably, the reference-free reward achieves performance comparable to the reference-guided approach, demonstrating that stable and spectrally consistent filtering strategies can be learned even without DNS data, drastically reducing the computational costs of the training phase. The proposed framework provides a robust and flexible alternative to manually tuned regularization parameters, enabling adaptive, physically consistent control of filtering in turbulent flow simulations.

Figures (19)

• Figure 1: Schematic time stepping mechanism in an RL-EF training episode, and related software.
• Figure 2: Action space in RL-EF.
• Figure 3: Test case 1. The two grids used for the simulations. The coarse grid is used for noEF, EF, and RL-EF simulations, while the fine mesh is used for the DNS reference simulation.
• Figure 4: Test case 1. Velocity magnitude fields for the filtered DNS, the noEF and EF with $\delta=\eta$. The fields are represented at different time instances.
• Figure 5: Test case 1. Cumulative reward and loss value during training steps, for the RL-EF methods.

Theorems & Definitions (6)

• Remark 2.1: Stokes filter
• Remark 2.2: Relaxation Step
• Remark 2.3: Classical choices for $\delta$
• Remark 3.1: Proximal Policy Optimization
• Remark 3.2
• Remark 3.3: Why not DF-RL-EF-rand?