Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Solving Linear-Quadratic Stochastic Control Problems with Signatures

Alif Aqsha, Peter Bank, Leandro Sánchez-Betancourt

TL;DR

To underpin a numerical approach based on truncated signatures, it is proved that the problem's value function can be approximated by finite-dimensional polynomial approximations when the truncation levels are chosen sufficiently high.

Abstract

We study a signature-driven numerical scheme to solve multi-dimensional linear-quadratic (LQ) stochastic control problems. Using that linear signature functionals are dense in the natural class of admissible controls, we show that our approach turns the original LQ problem into a deterministic convex quadratic polynomial optimisation. To underpin a numerical approach based on truncated signatures, we prove that the problem's value function can be approximated by finite-dimensional polynomial approximations when the truncation levels are chosen sufficiently high. Remarkably, our numerical experiments show very decent accuracy already for small truncation levels. Key tools for our analysis are (i) the algebraic representation of controlled stochastic differential equations and the associated cost function as linear functionals of the path signatures of the driving noise, (ii) the convergence of the truncated linear functionals, and (iii) the density of signature controls.

Solving Linear-Quadratic Stochastic Control Problems with Signatures

TL;DR

To underpin a numerical approach based on truncated signatures, it is proved that the problem's value function can be approximated by finite-dimensional polynomial approximations when the truncation levels are chosen sufficiently high.

Abstract

We study a signature-driven numerical scheme to solve multi-dimensional linear-quadratic (LQ) stochastic control problems. Using that linear signature functionals are dense in the natural class of admissible controls, we show that our approach turns the original LQ problem into a deterministic convex quadratic polynomial optimisation. To underpin a numerical approach based on truncated signatures, we prove that the problem's value function can be approximated by finite-dimensional polynomial approximations when the truncation levels are chosen sufficiently high. Remarkably, our numerical experiments show very decent accuracy already for small truncation levels. Key tools for our analysis are (i) the algebraic representation of controlled stochastic differential equations and the associated cost function as linear functionals of the path signatures of the driving noise, (ii) the convergence of the truncated linear functionals, and (iii) the density of signature controls.
Paper Structure (15 sections, 12 theorems, 68 equations, 2 figures)

This paper contains 15 sections, 12 theorems, 68 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Related literature
  3. Some notations for dealing with path signatures
  4. Tensor algebra notations
  5. Problem formulation
  6. Class of admissible tensor functionals
  7. Algebraic expression of a controlled multidimensional linear SDE
  8. Linear-quadratic cost functional and its algebraic properties
  9. Tensor representation of the cost functional
  10. Strict convexity of the cost functional
  11. Convergence of the truncated problem and density results
  12. Beyond Brownian noise
  13. Numerical experiments
  14. Brownian noise
  15. Fractional Brownian motion ($H=1/4$)

Key Result

Proposition 3.1

For any $\mathtt{p}^{(n)}, \mathtt{q}^{(n,n')} \in T^{\textup{ext}}((\mathbb{R}^{\bar{D}})^*)$, $n,n'=1,\dots,N$, with $(q^{(n,n')})^{\color{blue} \pmb{\phi}}=0$, there exists a unique $\mathtt{X}=(\mathtt{X}^{(1)},\cdots, \mathtt{X}^{(N)}) \in T^{\textup{ext}}((\mathbb{R}^{\bar{D}})^*)^N$ such that

Figures (2)

  • Figure 1: First and second panel: Estimation of $J(u)$ with 95% confidence interval (in this case $J(u^*) = 455$). Third and fourth panel: Estimation of $L^2(\mathrm{d} \mathbb{P} \otimes \mathrm{d} t)$ distance between $u^{{\mathrm{L}}, M}$ and the actual optimal control $u^*$ with 95% confidence interval.
  • Figure 2: First and second panel: Estimation of $J(u)$ with 95% confidence interval ($J(u^*) \approx 1.056$). Third and fourth panel: Estimation of $L^2(\mathrm{d} \mathbb{P} \otimes \mathrm{d} t)$ distance between $u^{{\mathrm{L}}, M}$ and the benchmark $u^*$ with 95% confidence interval.

Theorems & Definitions (22)

  • Proposition 3.1
  • Proof 1
  • Lemma 3.2
  • Proof 2
  • Lemma 3.3
  • Proof 3
  • Corollary 3.4
  • Proposition 3.6
  • Proof 4: Proof of Proposition \ref{['prop: cost functional as linear functional']}
  • Lemma 3.7
  • ...and 12 more