Scientific machine learning for closure models in multiscale problems: a review

TL;DR

This review maps the landscape of scientific machine learning for closure models in multiscale problems, detailing how physics-based dynamics can be augmented with data-driven closures through various reduced-model forms. It organizes methods by the degree of incorporated physics, objective-function strategies (apriori, aposteriori, reconstruction), and discretization considerations, while linking to inverse problems, Mori-Zwanzig theory, and multi-fidelity approaches. The authors highlight advances in physics-constrained learning, discretization-invariant architectures, and online learning, yet underscore persistent challenges in stability, interpretability, and generalization across regimes. The work also provides a thoughtful synthesis of non-locality, memory effects, and multiscale viewpoints, outlining a research agenda toward robust, interpretable closure models with reliable uncertainty quantification.

Abstract

Closure problems are omnipresent when simulating multiscale systems, where some quantities and processes cannot be fully prescribed despite their effects on the simulation's accuracy. Recently, scientific machine learning approaches have been proposed as a way to tackle the closure problem, combining traditional (physics-based) modeling with data-driven (machine-learned) techniques, typically through enriching differential equations with neural networks. This paper reviews the different reduced model forms, distinguished by the degree to which they include known physics, and the different objectives of a priori and a posteriori learning. The importance of adhering to physical laws (such as symmetries and conservation laws) in choosing the reduced model form and choosing the learning method is discussed. The effect of spatial and temporal discretization and recent trends toward discretization-invariant models are reviewed. In addition, we make the connections between closure problems and several other research disciplines: inverse problems, Mori-Zwanzig theory, and multi-fidelity methods. In conclusion, much progress has been made with scientific machine learning approaches for solving closure problems, but many challenges remain. In particular, the generalizability and interpretability of learned models is a major issue that needs to be addressed further.

Figures (3)

• Figure 1: A priori vs. a posteriori learning. Training data $\bm{u}$ and $\bar{\bm{u}}$ are obtained from solving the high-fidelity model $\bm{F}$. In blue: simulation of the high-fidelity model, from which the ground truth $\bar{\bm{u}}$ is derived. In yellow: training. In a posteriori learning, the loss function \ref{['eqn:Ltf']} includes evaluating the gradients of the reduced model, while in a priori learning, only the residual must be evaluated (equation \ref{['eqn:Lot_1']}).
• Figure 2: An example of imposing mass conservation for a 2D incompressible flow as a soft constraint (left) and hard constraint (right). For the hard constraint, the streamfunction $\psi$ is learned instead of the velocity field; the velocity field follows as $\bm{u} = \nabla \times \psi$, which by construction satisfies $\nabla \cdot \bm{u}= 0$.
• Figure 3: Optimize-then-discretize vs. discretize-then-optimize approaches. In orange: the optimization of a neural network to learn a time-continuous closure (left) or a time-discrete correction (right). In blue: the time integration/discretization step.