Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adversarial Adaptive Sampling: Unify PINN and Optimal Transport for the Approximation of PDEs

Kejun Tang, Jiayu Zhai, Xiaoliang Wan, Chao Yang

TL;DR

This work tackles solving PDEs with neural networks by addressing the statistical errors arising from random sampling in PINNs. It introduces Adversarial Adaptive Sampling (AAS), a minmax framework that jointly optimizes the PDE residual via a neural network and the training set via a generative sampler modeled as a normalizing flow, embedding Wasserstein-distance considerations to encourage a nearly uniform residual distribution and reduce MC variance. Theoretical results establish convergence properties and the practicality of a KRnet-based sampler, while numerical experiments on 2D peak problems and a 10D nonlinear PDE demonstrate improved accuracy and adaptive sampling behavior compared to baselines. The approach provides a principled link between PINNs and optimal transport, offering a scalable, variance-reducing strategy for high-dimensional PDEs with adaptive data generation.

Abstract

Solving partial differential equations (PDEs) is a central task in scientific computing. Recently, neural network approximation of PDEs has received increasing attention due to its flexible meshless discretization and its potential for high-dimensional problems. One fundamental numerical difficulty is that random samples in the training set introduce statistical errors into the discretization of loss functional which may become the dominant error in the final approximation, and therefore overshadow the modeling capability of the neural network. In this work, we propose a new minmax formulation to optimize simultaneously the approximate solution, given by a neural network model, and the random samples in the training set, provided by a deep generative model. The key idea is to use a deep generative model to adjust random samples in the training set such that the residual induced by the approximate PDE solution can maintain a smooth profile when it is being minimized. Such an idea is achieved by implicitly embedding the Wasserstein distance between the residual-induced distribution and the uniform distribution into the loss, which is then minimized together with the residual. A nearly uniform residual profile means that its variance is small for any normalized weight function such that the Monte Carlo approximation error of the loss functional is reduced significantly for a certain sample size. The adversarial adaptive sampling (AAS) approach proposed in this work is the first attempt to formulate two essential components, minimizing the residual and seeking the optimal training set, into one minmax objective functional for the neural network approximation of PDEs.

Adversarial Adaptive Sampling: Unify PINN and Optimal Transport for the Approximation of PDEs

TL;DR

This work tackles solving PDEs with neural networks by addressing the statistical errors arising from random sampling in PINNs. It introduces Adversarial Adaptive Sampling (AAS), a minmax framework that jointly optimizes the PDE residual via a neural network and the training set via a generative sampler modeled as a normalizing flow, embedding Wasserstein-distance considerations to encourage a nearly uniform residual distribution and reduce MC variance. Theoretical results establish convergence properties and the practicality of a KRnet-based sampler, while numerical experiments on 2D peak problems and a 10D nonlinear PDE demonstrate improved accuracy and adaptive sampling behavior compared to baselines. The approach provides a principled link between PINNs and optimal transport, offering a scalable, variance-reducing strategy for high-dimensional PDEs with adaptive data generation.

Abstract

Solving partial differential equations (PDEs) is a central task in scientific computing. Recently, neural network approximation of PDEs has received increasing attention due to its flexible meshless discretization and its potential for high-dimensional problems. One fundamental numerical difficulty is that random samples in the training set introduce statistical errors into the discretization of loss functional which may become the dominant error in the final approximation, and therefore overshadow the modeling capability of the neural network. In this work, we propose a new minmax formulation to optimize simultaneously the approximate solution, given by a neural network model, and the random samples in the training set, provided by a deep generative model. The key idea is to use a deep generative model to adjust random samples in the training set such that the residual induced by the approximate PDE solution can maintain a smooth profile when it is being minimized. Such an idea is achieved by implicitly embedding the Wasserstein distance between the residual-induced distribution and the uniform distribution into the loss, which is then minimized together with the residual. A nearly uniform residual profile means that its variance is small for any normalized weight function such that the Monte Carlo approximation error of the loss functional is reduced significantly for a certain sample size. The adversarial adaptive sampling (AAS) approach proposed in this work is the first attempt to formulate two essential components, minimizing the residual and seeking the optimal training set, into one minmax objective functional for the neural network approximation of PDEs.
Paper Structure (18 sections, 5 theorems, 54 equations, 6 figures, 1 table, 1 algorithm)

This paper contains 18 sections, 5 theorems, 54 equations, 6 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. PINN and its Statistical Errors
  3. Adversarial Adaptive Sampling
  4. A minmax formulation
  5. Understanding of AAS
  6. Implementation of AAS
  7. Related Work
  8. Numerical Results
  9. One-peak problem
  10. Two-peak problem
  11. High-dimensional nonlinear equation
  12. Conclusions
  13. Appendix
  14. Preliminaries from optimal transport theory
  15. The first convergence theorem and its proof
  16. ...and 3 more sections

Key Result

Theorem 1

Let $\mu$ be the Lebesgue measure on $\mathbb{R}^D$, which represents the uniform probability distribution on $\Omega$. In addition, we assume Assumption assumption2 holds. Then the optimal value of the min-max problem equation min-max is $0$. Moreover, there is a sequence $\{u_n\}_{n = 1}^{\infty}$ for some sequence of functions $\{p_n\}_{n = 1}^{\infty}$ satisfying the constraints in equation mi

Figures (6)

  • Figure 1: Two neural network models are simultaneously trained in the adversarial adaptive sampling framework. The residual is minimized and finally becomes "uniform", while the collocation points are updated and finally become nonuniform.
  • Figure 2: The results for the peak test problem. (a) The error behaviour. (b) The variance behavior. (c) The evolution of the training set.
  • Figure 3: The results for the two-peak test problem. (a) The exact solution. (b) AAS approximation. (c) The error behavior.
  • Figure 4: The evolution of the residual variance and the training set for the two-peak test problem. Left: The variance behavior. Right: The evolution of the training set.
  • Figure 5: The results of the ten-dimensional nonlinear test problem. (a) The error behavior. (b) The variance behaviour. (c) The evolution of the training set, $x_1 - x_2$ plane ($\beta = 10$).
  • ...and 1 more figures

Theorems & Definitions (8)

  • Theorem 1
  • Definition 2
  • Theorem 3: Kantorovich-Rubinstein theorem
  • Theorem 4
  • proof
  • Theorem
  • Lemma 5
  • proof