Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Solving Differential Equations with Constrained Learning

Viggo Moro, Luiz F. O. Chamon

TL;DR

Solving differential equations with constrained learning reframes PDE solution as a constrained learning task with worst-case losses, enabling seamless integration of mechanistic, structural, and observational knowledge. The authors prove the equivalence between weak PDE solutions and distributionally robust constrained objectives, and they develop a primal-dual MH-sampling algorithm that avoids heavy loss-weight tuning. The method, called science-constrained learning (SCL), delivers accurate solutions across a diverse set of PDEs and architectures, including parametric families and invariance constraints, often with competitive computational efficiency. This approach offers a robust, data- and knowledge-infused alternative to traditional mesh-based methods and standard PINNs, with practical implications for engineering and science workflows.

Abstract

(Partial) differential equations (PDEs) are fundamental tools for describing natural phenomena, making their solution crucial in science and engineering. While traditional methods, such as the finite element method, provide reliable solutions, their accuracy is often tied to the use of computationally intensive fine meshes. Moreover, they do not naturally account for measurements or prior solutions, and any change in the problem parameters requires results to be fully recomputed. Neural network-based approaches, such as physics-informed neural networks and neural operators, offer a mesh-free alternative by directly fitting those models to the PDE solution. They can also integrate prior knowledge and tackle entire families of PDEs by simply aggregating additional training losses. Nevertheless, they are highly sensitive to hyperparameters such as collocation points and the weights associated with each loss. This paper addresses these challenges by developing a science-constrained learning (SCL) framework. It demonstrates that finding a (weak) solution of a PDE is equivalent to solving a constrained learning problem with worst-case losses. This explains the limitations of previous methods that minimize the expected value of aggregated losses. SCL also organically integrates structural constraints (e.g., invariances) and (partial) measurements or known solutions. The resulting constrained learning problems can be tackled using a practical algorithm that yields accurate solutions across a variety of PDEs, neural network architectures, and prior knowledge levels without extensive hyperparameter tuning and sometimes even at a lower computational cost.

Solving Differential Equations with Constrained Learning

TL;DR

Solving differential equations with constrained learning reframes PDE solution as a constrained learning task with worst-case losses, enabling seamless integration of mechanistic, structural, and observational knowledge. The authors prove the equivalence between weak PDE solutions and distributionally robust constrained objectives, and they develop a primal-dual MH-sampling algorithm that avoids heavy loss-weight tuning. The method, called science-constrained learning (SCL), delivers accurate solutions across a diverse set of PDEs and architectures, including parametric families and invariance constraints, often with competitive computational efficiency. This approach offers a robust, data- and knowledge-infused alternative to traditional mesh-based methods and standard PINNs, with practical implications for engineering and science workflows.

Abstract

(Partial) differential equations (PDEs) are fundamental tools for describing natural phenomena, making their solution crucial in science and engineering. While traditional methods, such as the finite element method, provide reliable solutions, their accuracy is often tied to the use of computationally intensive fine meshes. Moreover, they do not naturally account for measurements or prior solutions, and any change in the problem parameters requires results to be fully recomputed. Neural network-based approaches, such as physics-informed neural networks and neural operators, offer a mesh-free alternative by directly fitting those models to the PDE solution. They can also integrate prior knowledge and tackle entire families of PDEs by simply aggregating additional training losses. Nevertheless, they are highly sensitive to hyperparameters such as collocation points and the weights associated with each loss. This paper addresses these challenges by developing a science-constrained learning (SCL) framework. It demonstrates that finding a (weak) solution of a PDE is equivalent to solving a constrained learning problem with worst-case losses. This explains the limitations of previous methods that minimize the expected value of aggregated losses. SCL also organically integrates structural constraints (e.g., invariances) and (partial) measurements or known solutions. The resulting constrained learning problems can be tackled using a practical algorithm that yields accurate solutions across a variety of PDEs, neural network architectures, and prior knowledge levels without extensive hyperparameter tuning and sometimes even at a lower computational cost.

Paper Structure

This paper contains 67 sections, 5 theorems, 64 equations, 18 figures, 10 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Boundary Value Problems
  4. Solving boundary value problems
  5. Science-Constrained Learning
  6. Mechanistic (PDE) knowledge.
  7. Structural knowledge.
  8. Observational knowledge.
  9. Science-constrained learning.
  10. Algorithm
  11. Experiments
  12. Solving a specific BVP
  13. Solving parametric families of BVPs
  14. Leveraging invariance when solving BVPs
  15. Supervised solution of BVPs
  16. ...and 52 more sections

Key Result

Proposition 3.1

Let $u^\dagger \in W^{k^\prime,2}(\mathcal{D})$, where $k^\prime \geq 1$ is the degree of the differential operator $D$, be such that $\sup_{\psi \in \mathcal{P}^2(\mathcal{D})} \mathop{\mathrm{\mathbb{E}}}\nolimits_{(x, t) \sim \psi} \![ (D [ u^\dagger ](x,t) - \tau(x, t) )^2 ] = 0$. If the dimen

Figures (18)

  • Figure 1: Solving parametric convection BVPs: (a) relative $L_2$ error vs. $\beta$ (legend reports number of differential operator evaluations per epoch). (b) Samples from $\psi_0^\text{PDE}$ at different training stages.
  • Figure 2: Using invariance in convection BVPs (BC and PDE losses in \ref{['eq:PINN']} and \ref{['eq:scl_problem']} are evaluated using a fixed set of collocation points).
  • Figure 2: Relative $L_2$ error on test set (average across 10 seeds, see App. \ref{['app:additional_experiments']} for standard deviation).
  • Figure 3: Final value of dual variables ($\lambda^\text{OB${}_j$}$) vs. magnitude of IC for Burgers' equation.
  • Figure 4: Relative $L_2$ error as a function of training epoch and differential operator evaluations for the parametric convection problem.
  • ...and 13 more figures

Theorems & Definitions (10)

  • Proposition 3.1
  • Proposition 4.1
  • Definition B.1: Sobolev space
  • Lemma B.2
  • proof : Proof of Lemma \ref{['thm:wasserstein']}
  • Definition B.3: Carathéodory integrand
  • Definition B.4: Decomposable space
  • Theorem B.5: Thm. 14.60 in Rockafellar04v
  • proof
  • Proposition D.4