Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An analysis of the derivative-free loss method for solving PDEs

Jihun Han, Yoonsang Lee

TL;DR

This work analyzes the derivative-free loss method (DFLM) for solving elliptic PDEs and fluid problems through a Feynman–Kac representation with stochastic walkers. It derives a bias bound for the empirical martingale loss that scales as $\Delta t / N_s$ and proves a problem-dependent lower bound $\Delta t^*$ necessary for effective learning, highlighting a trade-off between neighborhood exploration and estimator bias. The authors validate the theory with numerical experiments on a Poisson problem and the Taylor–Green vortex, demonstrating how to choose $\Delta t$ and $N_s$ to balance accuracy and computational cost. The findings offer practical guidance for deploying DFLM in high-dimensional PDEs and motivate extensions to multiscale settings with adaptive time-stepping strategies.

Abstract

This study analyzes the derivative-free loss method to solve a certain class of elliptic PDEs and fluid problems using neural networks. The approach leverages the Feynman-Kac formulation, incorporating stochastic walkers and their averaged values. We investigate how the time interval associated with the Feynman-Kac representation and the walker size influence computational efficiency, trainability, and sampling errors. Our analysis shows that the training loss bias scales proportionally with the time interval and the spatial gradient of the neural network, while being inversely proportional to the walker size. Moreover, we demonstrate that the time interval must be sufficiently long to enable effective training. These results indicate that the walker size can be chosen as small as possible, provided it satisfies the optimal lower bound determined by the time interval. Finally, we present numerical experiments that support our theoretical findings.

An analysis of the derivative-free loss method for solving PDEs

TL;DR

This work analyzes the derivative-free loss method (DFLM) for solving elliptic PDEs and fluid problems through a Feynman–Kac representation with stochastic walkers. It derives a bias bound for the empirical martingale loss that scales as and proves a problem-dependent lower bound necessary for effective learning, highlighting a trade-off between neighborhood exploration and estimator bias. The authors validate the theory with numerical experiments on a Poisson problem and the Taylor–Green vortex, demonstrating how to choose and to balance accuracy and computational cost. The findings offer practical guidance for deploying DFLM in high-dimensional PDEs and motivate extensions to multiscale settings with adaptive time-stepping strategies.

Abstract

This study analyzes the derivative-free loss method to solve a certain class of elliptic PDEs and fluid problems using neural networks. The approach leverages the Feynman-Kac formulation, incorporating stochastic walkers and their averaged values. We investigate how the time interval associated with the Feynman-Kac representation and the walker size influence computational efficiency, trainability, and sampling errors. Our analysis shows that the training loss bias scales proportionally with the time interval and the spatial gradient of the neural network, while being inversely proportional to the walker size. Moreover, we demonstrate that the time interval must be sufficiently long to enable effective training. These results indicate that the walker size can be chosen as small as possible, provided it satisfies the optimal lower bound determined by the time interval. Finally, we present numerical experiments that support our theoretical findings.
Paper Structure (9 sections, 6 theorems, 48 equations, 8 figures)

This paper contains 9 sections, 6 theorems, 48 equations, 8 figures.

Table of Contents

  1. Introduction
  2. Derivative-Free Neural Network Training Method
  3. Analysis
  4. Bias in the empirical martingale loss function
  5. Existence of a lower bound for the time interval of the Feynman-Kac formulation
  6. Numerical Experiments
  7. Poisson problem
  8. Taylor-Green vortex problem
  9. Discussions and conclusions

Key Result

Theorem 1

The empirical loss $\widetilde{\mathcal{L}}^{\Omega}(\bm{\theta})$ in Eq. eq:q_loss_discrete is a biased estimator of the exact martingale loss $\mathcal{L}^{\Omega}(\bm{\theta})$ in Eq. eq:q_loss_continuous. Moreover, when $u(\cdot;\bm{\theta})$ has a small PDE residual $\mathcal{N}[u(\cdot;\bm{\th

Figures (8)

  • Figure 1: Sampling diagram for DFLM
  • Figure 2: Empirical interior training loss for various walker size $N_s$ (horizontal axis) and time interval $\Delta t$ (different line types). (a) $m=1$ and (b) $m=3$.
  • Figure 3: Empirical interior training loss for various time interval $\Delta t$ (horizontal axis) and walker size $N_s$ (different line types). (a) $m=1$ and (b) $m=3$.
  • Figure 4: Relative $\mathcal{L}^2$ test error for varying time interval $\Delta t$. (a) $N_s=1$ (b) $N_s=400$.
  • Figure 5: Relative $\mathcal{L}^2$ test error as a function of time interval $\Delta t$ for various $N_s$ values. (a) $m=1$ and (b) $m=3$.
  • ...and 3 more figures

Theorems & Definitions (12)

  • Theorem 1
  • proof
  • Corollary 1
  • Lemma 1
  • proof
  • Remark
  • Example
  • Lemma 2
  • proof
  • Theorem 2
  • ...and 2 more