Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning Controlled Stochastic Differential Equations

Luc Brogat-Motte, Riccardo Bonalli, Alessandro Rudi

TL;DR

This work proposes a novel method for estimating both drift and diffusion coefficients of continuous, multidimensional, nonlinear controlled stochastic differential equations with non-uniform diffusion, using the Fokker-Planck equation.

Abstract

Identification of nonlinear dynamical systems is crucial across various fields, facilitating tasks such as control, prediction, optimization, and fault detection. Many applications require methods capable of handling complex systems while providing strong learning guarantees for safe and reliable performance. However, existing approaches often focus on simplified scenarios, such as deterministic models, known diffusion, discrete systems, one-dimensional dynamics, or systems constrained by strong structural assumptions such as linearity. This work proposes a novel method for estimating both drift and diffusion coefficients of continuous, multidimensional, nonlinear controlled stochastic differential equations with non-uniform diffusion. We assume regularity of the coefficients within a Sobolev space, allowing for broad applicability to various dynamical systems in robotics, finance, climate modeling, and biology. Leveraging the Fokker-Planck equation, we split the estimation into two tasks: (a) estimating system dynamics for a finite set of controls, and (b) estimating coefficients that govern those dynamics. We provide strong theoretical guarantees, including finite-sample bounds for \(L^2\), \(L^\infty\), and risk metrics, with learning rates adaptive to coefficients' regularity, similar to those in nonparametric least-squares regression literature. The practical effectiveness of our approach is demonstrated through extensive numerical experiments. Our method is available as an open-source Python library.

Learning Controlled Stochastic Differential Equations

TL;DR

This work proposes a novel method for estimating both drift and diffusion coefficients of continuous, multidimensional, nonlinear controlled stochastic differential equations with non-uniform diffusion, using the Fokker-Planck equation.

Abstract

Identification of nonlinear dynamical systems is crucial across various fields, facilitating tasks such as control, prediction, optimization, and fault detection. Many applications require methods capable of handling complex systems while providing strong learning guarantees for safe and reliable performance. However, existing approaches often focus on simplified scenarios, such as deterministic models, known diffusion, discrete systems, one-dimensional dynamics, or systems constrained by strong structural assumptions such as linearity. This work proposes a novel method for estimating both drift and diffusion coefficients of continuous, multidimensional, nonlinear controlled stochastic differential equations with non-uniform diffusion. We assume regularity of the coefficients within a Sobolev space, allowing for broad applicability to various dynamical systems in robotics, finance, climate modeling, and biology. Leveraging the Fokker-Planck equation, we split the estimation into two tasks: (a) estimating system dynamics for a finite set of controls, and (b) estimating coefficients that govern those dynamics. We provide strong theoretical guarantees, including finite-sample bounds for , , and risk metrics, with learning rates adaptive to coefficients' regularity, similar to those in nonparametric least-squares regression literature. The practical effectiveness of our approach is demonstrated through extensive numerical experiments. Our method is available as an open-source Python library.

Paper Structure

This paper contains 129 sections, 45 theorems, 272 equations, 23 figures.

Table of Contents

  1. Abstract.
  2. Keywords.
  3. Mathematics Subject Classification.
  4. Introduction
  5. Contributions.
  6. Related work
  7. System identification.
  8. Online learning in system identification.
  9. Uncontrolled SDE estimation.
  10. Controlled SDE estimation.
  11. Paper organisation
  12. Paper notations
  13. Problem setting
  14. Controlled SDE identification.
  15. Probability density associated with $(b, \sigma^2)$.
  16. ...and 114 more sections

Key Result

Lemma 3.1

Let $\left(\mathcal{L}^{(b, \sigma)(u)}\right)^*$ denotes the dual of the Kolmogorov generator Under Assumptions as:min_reg and as:uniform_ellipticity, the strong Fokker-Planck equation holds, namely,

Figures (23)

  • Figure 1: OU process. True and estimated probability densities, $p(t,x)$(left) and $\hat{p}(t,x)$(right), w.r.t $x$ for several $t \in [0, 10]$.
  • Figure 2: OU process. Estimated coefficients $\hat{b}(t,x)$(left) and $\hat{\sigma}(t, x)^2$(right) w.r.t. $x$ for several $t \in [0, 10]$.
  • Figure 3: OU process. 100 samples from the SDEs associated with the true (left) and estimated (right) coefficients.
  • Figure 6: Training set of 10 i.i.d. piecewise-constant controls.
  • Figure 8: Training set of 20 i.i.d. sinusoidal controls.
  • ...and 18 more figures

Theorems & Definitions (86)

  • Remark 1: Non-identifiability of SDE coefficients
  • Example 1: Prototypical non-identifiable SDE coefficients
  • Lemma 3.1: SFP equation
  • Lemma 3.2: FP matching inequality
  • Theorem 4.1: $L^2$ Learning rates
  • Remark 2: Dependency in $K$ and $N$
  • Remark 3: Dependency in the density estimation
  • Remark 4: Control-dependent sampling of $[0, T] \times \mathbb{R}^n$
  • Remark 5: Soft shape Constraint
  • Remark 6: Decomposition of the embedding property
  • ...and 76 more