Finite-Horizon LQR for General Markov Jump Linear Systems: Deterministic Reformulation and Reduced-Order Solution

TL;DR

This work tackles the finite-horizon LQR problem for continuous-time Markov Jump Linear Systems with general finite-state Markov chains that may include transient or absorbing states and non-communicating classes. It develops a deterministic reformulation based on second-moment matrices $X_i(t)=\mathscr{E}(x(t)x(t)'\cdot 1_{\{\theta(t)=i\}})$ and introduces a visited-states projection, yielding a reduced-order description on $\mathbb{V}$ that excludes unvisited modes. The core contributions are the rigorous derivation of a Hamilton–Jacobi–Bellman equation in a Hilbert-space setting via the Riesz–Fréchet representation, and a reduced-order system of coupled Riccati differential equations for the visited states $\mathscr{Z}$ that completely characterizes the optimal policy with $L_i^*(t)=-R_i^{-1}B_i'(t)Y_i(t)$. The theory is validated numerically on systems with non-communicating states and an absorbing satellite-failure mode, demonstrating accurate cost computation and substantial order reduction, with potential impact for fault-tolerant control and regime-switching applications where irreducibility cannot be assumed.

Abstract

This paper studies the Linear Quadratic Regulator (LQR) problem for continuous-time Markov Jump Linear Systems (MJLS) governed by general finite-state Markov chains that may include transient, absorbing, or non-communicating states. The problem, posed over a finite time horizon, is reformulated deterministically by expressing the cost functional in terms of a collection of second-moment matrices of the system state. A projection operator is introduced to restrict the analysis to the subspace of visited states, namely those with positive probability of being reached within the time horizon. The solution of the resulting deterministic problem is obtained from a reduced-order system of coupled matrix Riccati differential equations involving only the visited states, which define a quadratic value function satisfying a Hamilton-Jacobi-Bellman type equation. The structure of this equation is formally justified in the matrix setting via the Riesz-Frechet representation theorem, establishing a rigorous foundation for the deterministic reformulation and resolving an open aspect in previous literature. Several numerical examples, including cases with non-communicating states, validate the theoretical results and illustrate the practical relevance of the proposed generalization.

Theorems & Definitions (5)

• proof
• proof
• proof
• proof
• proof