Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive deep density approximation for stochastic dynamical systems

Junjie He, Qifeng Liao, Xiaoliang Wan

TL;DR

This work tackles uncertainty quantification for stochastic dynamical systems by learning time-evolving PDFs from the Liouville equation using a time-dependent normalizing-flow, the temporal KRnet (tKRnet). It integrates adaptive collocation sampling with a physics-informed loss based on the log-Liouville residual and introduces temporal decomposition to address long-time integration, all while providing a KL-divergence bound that links residuals to approximation quality. The framework demonstrates strong performance on four challenging problems, including high-dimensional Lorenz-96, with accurate PDFs, means, and variances and clear improvements through adaptive iterations and temporal strategies. The approach yields a scalable, explicit density model suitable for high-dimensional UQ tasks in complex dynamical systems, enabling efficient PDF evaluation and sampling at arbitrary times $t$.

Abstract

In this paper we consider adaptive deep neural network approximation for stochastic dynamical systems. Based on the Liouville equation associated with the stochastic dynamical systems, a new temporal KRnet (tKRnet) is proposed to approximate the probability density functions (PDFs) of the state variables. The tKRnet gives an explicit density model for the solution of the Liouville equation, which alleviates the curse of dimensionality issue that limits the application of traditional grid based numerical methods. To efficiently train the tKRnet, an adaptive procedure is developed to generate collocation points for the corresponding residual loss function, where samples are generated iteratively using the approximate density function at each iteration. A temporal decomposition technique is also employed to improve the long-time integration. Theoretical analysis of our proposed method is provided, and numerical examples are presented to demonstrate its performance.

Adaptive deep density approximation for stochastic dynamical systems

TL;DR

This work tackles uncertainty quantification for stochastic dynamical systems by learning time-evolving PDFs from the Liouville equation using a time-dependent normalizing-flow, the temporal KRnet (tKRnet). It integrates adaptive collocation sampling with a physics-informed loss based on the log-Liouville residual and introduces temporal decomposition to address long-time integration, all while providing a KL-divergence bound that links residuals to approximation quality. The framework demonstrates strong performance on four challenging problems, including high-dimensional Lorenz-96, with accurate PDFs, means, and variances and clear improvements through adaptive iterations and temporal strategies. The approach yields a scalable, explicit density model suitable for high-dimensional UQ tasks in complex dynamical systems, enabling efficient PDF evaluation and sampling at arbitrary times .

Abstract

In this paper we consider adaptive deep neural network approximation for stochastic dynamical systems. Based on the Liouville equation associated with the stochastic dynamical systems, a new temporal KRnet (tKRnet) is proposed to approximate the probability density functions (PDFs) of the state variables. The tKRnet gives an explicit density model for the solution of the Liouville equation, which alleviates the curse of dimensionality issue that limits the application of traditional grid based numerical methods. To efficiently train the tKRnet, an adaptive procedure is developed to generate collocation points for the corresponding residual loss function, where samples are generated iteratively using the approximate density function at each iteration. A temporal decomposition technique is also employed to improve the long-time integration. Theoretical analysis of our proposed method is provided, and numerical examples are presented to demonstrate its performance.
Paper Structure (17 sections, 1 theorem, 56 equations, 13 figures, 1 algorithm)

This paper contains 17 sections, 1 theorem, 56 equations, 13 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem setup and preliminaries
  3. Temporal KRnet (tKRnet)
  4. Time-dependent affine coupling layers
  5. Scale-bias, squeezing and nonlinear layers
  6. The overall structure of tKRnet
  7. Adaptive sampling based physics-informed training for density approximation
  8. Physics-informed training and adaptive sampling
  9. Temporal decomposition for long-time integration
  10. Theoretical properties
  11. Numerical experiments
  12. Double gyre flow
  13. Kraichnan-Orszag
  14. Forced Duffing oscillator
  15. Lorenz-96 system
  16. ...and 2 more sections

Key Result

Theorem 4.1

\newlabelthm:time-derivative-KL0 Assume $p(\boldsymbol{x},t)\rightarrow 0, p_{\Theta}(\boldsymbol{x},t)\rightarrow 0$ as $\|\boldsymbol{x}\|_2\rightarrow \infty$. Denote $\Omega_{C}=\{\boldsymbol{x}|\|\boldsymbol{x}\|_2\leq C\}$ with $C>0$. For any $t\in[0,T]$, assume $\lim_{C\rightarrow \infty} \

Figures (13)

  • Figure 1: Overall structure of tKRnet.
  • Figure 1: Double gyre flow problem for $t\in(0,5]$: the reference solution and the tKRnet solution.
  • Figure 1: ODE based approximation.
  • Figure 2: Double gyre flow problem for $t\in(0,5]$: error and KL divergence values of the tKRnet solutions.
  • Figure 2: Relative errors of ODE based approximation and PDE based approximation.
  • ...and 8 more figures

Theorems & Definitions (3)

  • Theorem 4.1: Control of KL divergence
  • Proof 1
  • Remark 4.2