Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Linear Quadratic Control with Non-Markovian and Non-Semimartingale Noise Models

Mostafa M. Shibl, Sharan Srinivasan, Harsha Honnappa, Vijay Gupta

TL;DR

This work extends linear-quadratic control to non-Markovian and non-semimartingale disturbances by adopting rough path theory, formulating a generalized LQ problem (gLQ) with rough process and observation noises. It derives a Riccati-based optimal controller augmented by a noise-prediction term V(t) and, for partial-state information, a rough Kalman-Bucy-type observer, establishing pathwise optimality and a rough-path-enabled separation principle. The authors prove existence and structure of the solution and illustrate substantial robustness improvements over classical LQR in simulations driven by fractional Brownian motion and stable Lévy noise. The approach offers pathwise guarantees and suggests future integration with path signatures and reinforcement learning for path-dependent control under rough disturbances.

Abstract

The standard linear quadratic Gaussian (LQG) framework assumes a Brownian noise process and relies on classical stochastic calculus tools, such as those based on Itô calculus. In this paper, we solve a generalized linear quadratic optimal control problem where the process and measurement noises can be non-Markovian and non-semimartingale stochastic processes with sample paths that have low Hölder regularity. Since these noise models do not, in general, permit the use of the standard Itô calculus, we employ rough path theory to formulate and solve the problem. By leveraging signature representations and controlled rough paths, we derive the optimal state estimation and control strategies.

Linear Quadratic Control with Non-Markovian and Non-Semimartingale Noise Models

TL;DR

This work extends linear-quadratic control to non-Markovian and non-semimartingale disturbances by adopting rough path theory, formulating a generalized LQ problem (gLQ) with rough process and observation noises. It derives a Riccati-based optimal controller augmented by a noise-prediction term V(t) and, for partial-state information, a rough Kalman-Bucy-type observer, establishing pathwise optimality and a rough-path-enabled separation principle. The authors prove existence and structure of the solution and illustrate substantial robustness improvements over classical LQR in simulations driven by fractional Brownian motion and stable Lévy noise. The approach offers pathwise guarantees and suggests future integration with path signatures and reinforcement learning for path-dependent control under rough disturbances.

Abstract

The standard linear quadratic Gaussian (LQG) framework assumes a Brownian noise process and relies on classical stochastic calculus tools, such as those based on Itô calculus. In this paper, we solve a generalized linear quadratic optimal control problem where the process and measurement noises can be non-Markovian and non-semimartingale stochastic processes with sample paths that have low Hölder regularity. Since these noise models do not, in general, permit the use of the standard Itô calculus, we employ rough path theory to formulate and solve the problem. By leveraging signature representations and controlled rough paths, we derive the optimal state estimation and control strategies.

Paper Structure

This paper contains 14 sections, 7 theorems, 87 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Rough Path Theory Preliminaries
  3. Model
  4. Main Results
  5. Optimal Controller with State Observation
  6. Optimal Controller without State Observation
  7. Illustrative Examples
  8. Fractional Brownian Motion
  9. Lévy Processes (Stable Distribution)
  10. Conclusion
  11. Supporting Theorems and Lemmas
  12. Preliminaries
  13. Fractional Brownian Motion
  14. Stable Distribution

Key Result

Lemma 1

Let $\boldsymbol{\zeta}=(\zeta,\zeta^{(2)})$ be a geometric $p$-rough path on $[0,T]$ with $p\in(2,3)$. Let $\{\boldsymbol{\eta}^n\}_{n\ge1}$ be any sequence of lifted smooth paths $\boldsymbol{\eta}^n$ whose lifts $(\eta^n,(\eta^n)^{(2)})$ converge to $\boldsymbol{\zeta}$ in the $1/p$-Hölder rough For each smooth $\eta^n$, let $\Psi^{\eta^n}$ be the solution of the classical HJB equation. We say

Figures (4)

  • Figure 1: State trajectory under fBm noises with classical LQ controller
  • Figure 2: State trajectory under fBm noises with rough path gLQ controller
  • Figure 3: State trajectory under Stable distribution noises with classical LQ controller
  • Figure 4: State trajectory under Stable distribution noises with rough path gLQ controller

Theorems & Definitions (15)

  • Definition 1: Rough Paths book
  • Definition 2: Hölder Regularity of a Rough Path book
  • Definition 3: Geometric $p$-rough path
  • Lemma 1: Definition 3.13 in allan
  • Theorem 2: Existence & Optimality of gLQ Controller
  • proof
  • Corollary 3: Pathwise Optimality of gLQ Controller
  • proof
  • Theorem 4: Observer Form of Correlated Noise Case
  • proof
  • ...and 5 more