Reinforcement Learning Closures for Underresolved Partial Differential Equations using Synthetic Data

TL;DR

The paper tackles the challenge of solving underresolved PDEs efficiently by learning closures for coarse-grained simulations. It leverages synthetic data generated via the method of manufactured solutions (MMS) to train a reinforcement learning–based closure model that augments coarse-grained dynamics toward the high-fidelity behavior. The framework is demonstrated on the 1D Burgers' equation, the 2D Burgers' equation, and the 2D advection equation, showing strong in-distribution and extrapolative out-of-distribution performance, with notable improvements over a neural-operator baseline in extrapolation. These results suggest a data-efficient, computationally cheaper path to accurate closures for systems with scarce data, while acknowledging limitations and outlining future directions such as sensitivity analysis and integration with differentiable solvers.

Abstract

Partial Differential Equations (PDEs) describe phenomena ranging from turbulence and epidemics to quantum mechanics and financial markets. Despite recent advances in computational science, solving such PDEs for real-world applications remains prohibitively expensive because of the necessity of resolving a broad range of spatiotemporal scales. In turn, practitioners often rely on coarse-grained approximations of the original PDEs, trading off accuracy for reduced computational resources. To mitigate the loss of detail inherent in such approximations, closure models are employed to represent unresolved spatiotemporal interactions. We present a framework for developing closure models for PDEs using synthetic data acquired through the method of manufactured solutions. These data are used in conjunction with reinforcement learning to provide closures for coarse-grained PDEs. We illustrate the efficacy of our method using the one-dimensional and two-dimensional Burgers' equations and the two-dimensional advection equation. Moreover, we demonstrate that closure models trained for inhomogeneous PDEs can be effectively generalized to homogeneous PDEs. The results demonstrate the potential for developing accurate and computationally efficient closure models for systems with scarce data.

Figures (18)

• Figure 1: Illustration of the RL framework with the agents embedded in the CGS and their action $A$ computed by a policy $\pi$. The CGS solution is computed by using the update operator $\mathcal{F}$ and afterwards modified by the agents in order to bring the solution closer to the FGS. The reward $R$ measures how much the action taken improves the CGS.
• Figure 2: Results for the inhomogeneous 1D Burgers' equation. The mean squared error (MSE) is calculated with respect to the proposed analytical MMS solution. The plots show median values for the CGS () and the Closure-RL () across 30 different MMS solutions. The shaded regions around the medians represent the interquartile range (25th to 75th percentile).
• Figure 3: Visualization of the evolution of $\psi$ for the inhomogeneous 1D Burgers' equation at five different time snapshots. The figure shows the evolution for the CGS (), the Closure-RL simulation (), and the MMS solution ($\cdots$).
• Figure 4: Results for the homogeneous 1D Burgers' equation. The MSE is calculated with respect to the FGS. The plots show median values for the CGS () and the Closure-RL () across 30 different MMS solutions. The shaded regions around the medians represent the interquartile range (25th to 75th percentile).
• Figure 5: Visualization of the evolution of $\psi$ for the homogeneous 1D Burgers' equation at five different time snapshots. The figure shows the evolution for the CG (), the Closure-RL (), and the FG ($\cdots$) simulations.