Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Universal approximation property of neural stochastic differential equations

Anna P. Kwossek, David J. Prömel, Josef Teichmann

TL;DR

The paper develops a universal approximation framework for neural SDEs by proving that several neural network classes with a linear growth constraint can approximate continuous functions locally uniformly. This universality is then transferred to neural SDEs, showing that neural SDEs can approximate general SDEs under standard regularity, with quantitative error bounds in both Lipschitz and Hölder settings. The approach combines weighted function spaces, constructive approximation with ReLU and squashing activations, and growth-controlled bounds to establish a rigorous pathway from neural network universality to stochastic dynamical systems. The results enable flexible, time-continuous modeling of stochastic processes with provable approximation guarantees, with potential impact on data-driven stochastic modeling and financial engineering.

Abstract

We identify various classes of neural networks that are able to approximate continuous functions locally uniformly subject to fixed global linear growth constraints. For such neural networks the associated neural stochastic differential equations can approximate general stochastic differential equations, both of Itô diffusion type, arbitrarily well. Moreover, quantitative error estimates are derived for stochastic differential equations with sufficiently regular coefficients.

Universal approximation property of neural stochastic differential equations

TL;DR

The paper develops a universal approximation framework for neural SDEs by proving that several neural network classes with a linear growth constraint can approximate continuous functions locally uniformly. This universality is then transferred to neural SDEs, showing that neural SDEs can approximate general SDEs under standard regularity, with quantitative error bounds in both Lipschitz and Hölder settings. The approach combines weighted function spaces, constructive approximation with ReLU and squashing activations, and growth-controlled bounds to establish a rigorous pathway from neural network universality to stochastic dynamical systems. The results enable flexible, time-continuous modeling of stochastic processes with provable approximation guarantees, with potential impact on data-driven stochastic modeling and financial engineering.

Abstract

We identify various classes of neural networks that are able to approximate continuous functions locally uniformly subject to fixed global linear growth constraints. For such neural networks the associated neural stochastic differential equations can approximate general stochastic differential equations, both of Itô diffusion type, arbitrarily well. Moreover, quantitative error estimates are derived for stochastic differential equations with sufficiently regular coefficients.

Paper Structure

This paper contains 9 sections, 10 theorems, 91 equations.

Table of Contents

  1. Introduction
  2. Universal approximation property under a linear growth constraint
  3. Linearly activating activation functions
  4. Combining the ReLU activation function and a general activation function
  5. The ReLU activation function
  6. Two activation functions: the ReLU activation function and a squashing activation function
  7. Universal approximation property of neural SDEs
  8. Quantitative approximation results for SDEs with Lipschitz continuous coefficients
  9. Quantitative approximation results for SDEs with Hölder continuous diffusion coefficient

Key Result

Theorem 2.3

If the activation function $\varrho \in C(\mathbb{R};\mathbb{R})$ is linearly activating, then $\mathcal{N\mkern-2mu N}^\varrho_{k+1; n_1 \times n_2}$ has the universal approximation property under a linear growth constraint in the sense of Definition def: approximation property. Moreover, the const

Theorems & Definitions (26)

  • Definition 2.1
  • Remark 2.2
  • Theorem 2.3
  • proof
  • Proposition 2.4
  • Remark 2.5
  • proof : Proof of Proposition \ref{['prop: UAP with ReLU in weighted space']}
  • Corollary 2.6
  • Remark 2.7
  • Remark 2.8
  • ...and 16 more