Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Integration Matters for Learning PDEs with Backwards SDEs

Sungje Park, Stephen Tu

TL;DR

This work identifies a discretization bias in EM-based BSDE losses that prevents BSDE solvers from matching PINN performance on high-dimensional PDEs. By reframing the forward SDE in Stratonovich form and employing the stochastic Heun integrator, the authors eliminate the dominant one-step bias and restore competitive accuracy with PINNs, even for single-step self-consistency losses. The approach yields robust results across multiple high-dimensional benchmarks, though it incurs higher computational cost and requires careful numerical handling (e.g., Hessian terms). The findings underscore integration scheme choice as a critical algorithmic detail in BSDE-based PDE solvers and open avenues for efficiency and stability enhancements in model-free or high-dimensional settings.

Abstract

Backward stochastic differential equation (BSDE)-based deep learning methods provide an alternative to Physics-Informed Neural Networks (PINNs) for solving high-dimensional partial differential equations (PDEs), offering potential algorithmic advantages in settings such as stochastic optimal control, where the PDEs of interest are tied to an underlying dynamical system. However, standard BSDE-based solvers have empirically been shown to underperform relative to PINNs in the literature. In this paper, we identify the root cause of this performance gap as a discretization bias introduced by the standard Euler-Maruyama (EM) integration scheme applied to one-step self-consistency BSDE losses, which shifts the optimization landscape off target. We find that this bias cannot be satisfactorily addressed through finer step-sizes or multi-step self-consistency losses. To properly handle this issue, we propose a Stratonovich-based BSDE formulation, which we implement with stochastic Heun integration. We show that our proposed approach completely eliminates the bias issues faced by EM integration. Furthermore, our empirical results show that our Heun-based BSDE method consistently outperforms EM-based variants and achieves competitive results with PINNs across multiple high-dimensional benchmarks. Our findings highlight the critical role of integration schemes in BSDE-based PDE solvers, an algorithmic detail that has received little attention thus far in the literature.

Integration Matters for Learning PDEs with Backwards SDEs

TL;DR

This work identifies a discretization bias in EM-based BSDE losses that prevents BSDE solvers from matching PINN performance on high-dimensional PDEs. By reframing the forward SDE in Stratonovich form and employing the stochastic Heun integrator, the authors eliminate the dominant one-step bias and restore competitive accuracy with PINNs, even for single-step self-consistency losses. The approach yields robust results across multiple high-dimensional benchmarks, though it incurs higher computational cost and requires careful numerical handling (e.g., Hessian terms). The findings underscore integration scheme choice as a critical algorithmic detail in BSDE-based PDE solvers and open avenues for efficiency and stability enhancements in model-free or high-dimensional settings.

Abstract

Backward stochastic differential equation (BSDE)-based deep learning methods provide an alternative to Physics-Informed Neural Networks (PINNs) for solving high-dimensional partial differential equations (PDEs), offering potential algorithmic advantages in settings such as stochastic optimal control, where the PDEs of interest are tied to an underlying dynamical system. However, standard BSDE-based solvers have empirically been shown to underperform relative to PINNs in the literature. In this paper, we identify the root cause of this performance gap as a discretization bias introduced by the standard Euler-Maruyama (EM) integration scheme applied to one-step self-consistency BSDE losses, which shifts the optimization landscape off target. We find that this bias cannot be satisfactorily addressed through finer step-sizes or multi-step self-consistency losses. To properly handle this issue, we propose a Stratonovich-based BSDE formulation, which we implement with stochastic Heun integration. We show that our proposed approach completely eliminates the bias issues faced by EM integration. Furthermore, our empirical results show that our Heun-based BSDE method consistently outperforms EM-based variants and achieves competitive results with PINNs across multiple high-dimensional benchmarks. Our findings highlight the critical role of integration schemes in BSDE-based PDE solvers, an algorithmic detail that has received little attention thus far in the literature.
Paper Structure (51 sections, 29 theorems, 174 equations, 9 figures, 4 tables, 3 algorithms)

This paper contains 51 sections, 29 theorems, 174 equations, 9 figures, 4 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Background and Problem Setup
  4. Physics-Informed Neural Networks (PINNs).
  5. Backwards SDEs and self-consistency losses.
  6. Euler-Maruyama integration.
  7. Analysis of One-Step Self-Consistency Losses
  8. The Hölder space $C^{k,1}$.
  9. Analysis of Euler-Maruyama for BSDE
  10. Stratonovich BSDEs and Stochastic Heun Integration
  11. Trade-offs for Multi-Step BSDE Losses
  12. BSDE loss and Euler-Maruyama discretization.
  13. Stratonovich BSDE and Heun discretization.
  14. Experiments
  15. Efficient sub-sampling BSDE implementation.
  16. ...and 36 more sections

Key Result

Lemma 4.0

Suppose that $f, g$ are bounded and $u_\theta$ is $C^{2,1}$. We have that where the $O(\cdot)$ hides factors depending on $d$, the bounds on $f, g$, and $\lVert u_\theta \rVert_{C^{2,1}}$.

Figures (9)

  • Figure 1: A plot of both $L_{\mathrm{EM},\tau}(\theta)$ and $L_{\mathrm{Heun},\tau}(\theta)$ at various levels of discretization. The PDE is a one dimensional Linear Quadratic Regulator HJB equation, where $\theta$ parameterizes a quadratic value function.
  • Figure 2: A plot of the 100D HJB reference and learned solutions for each model and the associated RL2 errors.
  • Figure 2: A table of average training time overhead relative to PINNs for both the full and batched algorithm runs.
  • Figure 3: RL2 performance for 10D BSB at discretization step-sizes $\tau=N^{-1}$ for $N \in \{25, 50, 100, 200\}$.
  • Figure 4: A plot of the RL2 performance versus runtime for the 100D HJB problem.
  • ...and 4 more figures

Theorems & Definitions (53)

  • Lemma 4.0
  • Theorem 4.0
  • Lemma 4.0
  • Theorem 4.0
  • Proposition D.1
  • proof
  • Proposition D.2
  • proof
  • Proposition D.3
  • proof
  • ...and 43 more