Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep adaptive sampling for surrogate modeling without labeled data

Xili Wang, Kejun Tang, Jiayu Zhai, Xiaoliang Wan, Chao Yang

TL;DR

This work tackles surrogate modeling for parametric differential equations in the absence of labeled data. It introduces $\mathrm{DAS}^2$, a physics-informed framework that uses a KRnet-based deep generative model to approximate the residual-induced sampling distribution and to adaptively generate collocation points in both space and parameter domains, thereby reducing discretization (statistical) error for low-regularity problems. The method is validated across multiple settings, including a parametric ODE, high-dimensional operator learning, a geometry-driven optimal control problem, and parametric lid-driven cavity flows, consistently outperforming uniform, RAR, and QRS strategies with faster convergence and competitive inference times. The results demonstrate that deep generative, residual-guided sampling can significantly improve all-at-once surrogates for high-dimensional, low-regularity PDEs without needing labeled simulation data, offering a practical path to rapid, reliable uncertainty quantification and design under parametric variation.

Abstract

Surrogate modeling is of great practical significance for parametric differential equation systems. In contrast to classical numerical methods, using physics-informed deep learning methods to construct simulators for such systems is a promising direction due to its potential to handle high dimensionality, which requires minimizing a loss over a training set of random samples. However, the random samples introduce statistical errors, which may become the dominant errors for the approximation of low-regularity and high-dimensional problems. In this work, we present a deep adaptive sampling method for surrogate modeling ($\text{DAS}^2$), where we generalize the deep adaptive sampling (DAS) method [62] [Tang, Wan and Yang, 2023] to build surrogate models for low-regularity parametric differential equations. In the parametric setting, the residual loss function can be regarded as an unnormalized probability density function (PDF) of the spatial and parametric variables. This PDF is approximated by a deep generative model, from which new samples are generated and added to the training set. Since the new samples match the residual-induced distribution, the refined training set can further reduce the statistical error in the current approximate solution. We demonstrate the effectiveness of $\text{DAS}^2$ with a series of numerical experiments, including the parametric lid-driven 2D cavity flow problem with a continuous range of Reynolds numbers from 100 to 1000.

Deep adaptive sampling for surrogate modeling without labeled data

TL;DR

This work tackles surrogate modeling for parametric differential equations in the absence of labeled data. It introduces , a physics-informed framework that uses a KRnet-based deep generative model to approximate the residual-induced sampling distribution and to adaptively generate collocation points in both space and parameter domains, thereby reducing discretization (statistical) error for low-regularity problems. The method is validated across multiple settings, including a parametric ODE, high-dimensional operator learning, a geometry-driven optimal control problem, and parametric lid-driven cavity flows, consistently outperforming uniform, RAR, and QRS strategies with faster convergence and competitive inference times. The results demonstrate that deep generative, residual-guided sampling can significantly improve all-at-once surrogates for high-dimensional, low-regularity PDEs without needing labeled simulation data, offering a practical path to rapid, reliable uncertainty quantification and design under parametric variation.

Abstract

Surrogate modeling is of great practical significance for parametric differential equation systems. In contrast to classical numerical methods, using physics-informed deep learning methods to construct simulators for such systems is a promising direction due to its potential to handle high dimensionality, which requires minimizing a loss over a training set of random samples. However, the random samples introduce statistical errors, which may become the dominant errors for the approximation of low-regularity and high-dimensional problems. In this work, we present a deep adaptive sampling method for surrogate modeling (), where we generalize the deep adaptive sampling (DAS) method [62] [Tang, Wan and Yang, 2023] to build surrogate models for low-regularity parametric differential equations. In the parametric setting, the residual loss function can be regarded as an unnormalized probability density function (PDF) of the spatial and parametric variables. This PDF is approximated by a deep generative model, from which new samples are generated and added to the training set. Since the new samples match the residual-induced distribution, the refined training set can further reduce the statistical error in the current approximate solution. We demonstrate the effectiveness of with a series of numerical experiments, including the parametric lid-driven 2D cavity flow problem with a continuous range of Reynolds numbers from 100 to 1000.
Paper Structure (19 sections, 2 theorems, 53 equations, 17 figures, 2 tables, 2 algorithms)

This paper contains 19 sections, 2 theorems, 53 equations, 17 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Main contributions
  3. Related work
  4. Adaptive sampling methods
  5. Neural network methods for parametric differential equations
  6. Problem setting and statistical errors in physics-informed surrogate modeling
  7. Deep adaptive sampling for surrogate modeling
  8. Sample from a joint PDF
  9. Sample from a marginal PDF
  10. Algorithm
  11. Analysis
  12. Numerical study
  13. A one-dimensional parametric ordinary differential equation
  14. Operator learning for a dynamical system with high-dimensional parameters
  15. Surrogate modeling for an optimal control problem with geometrical parametrization
  16. ...and 4 more sections

Key Result

Theorem 1

Suppose that Assumption assump_lip_operator and Assumption assump_bounded are satisfied and the boundary loss is zero. Let $\boldsymbol{\theta}_N^*$ be a minimizer of $J_{r,N}$ where the collocation points are independently drawn from a given probability distribution. Given $\varepsilon \in (0,1)$, with probability at least $1 - (4a \mathfrak{L}/\varepsilon^2)^D \mathrm{exp}(-N_r \varepsilon^4/2c

Figures (17)

  • Figure 1: The errors for the parametric ODE test problem.
  • Figure 2: The results of the parametric ODE test problem. Left: The evolution of $\mathsf{S}_{\Omega,k}^g$ in $\mathrm{DAS}^2$; Right: The exact solution and the approximate solutions with different sampling strategies for $\xi = 2.725$.
  • Figure 3: Approximation errors for the operator learning problem.
  • Figure 4: The error evolution of $\mathrm{DAS}^2$ at different adaptivity iteration steps for the operator learning problem. $|\mathsf{S}_{\Omega_p}|=1\times 10^5.$
  • Figure 5: The evolution of $\mathsf{S}_{\Omega_p,k}^g$ in $\mathrm{DAS}^2$ for the operator learning problem, $|\mathsf{S}_{\Omega_p}|=1\times 10^5$.
  • ...and 12 more figures

Theorems & Definitions (4)

  • Theorem 1
  • Theorem 2
  • proof
  • proof