Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LQR for Systems with Probabilistic Parametric Uncertainties: A Gradient Method

Leilei Cui, Richard D. Braatz

Abstract

A gradient-based method is proposed for solving the linear quadratic regulator (LQR) problem for linear systems with nonlinear dependence on time-invariant probabilistic parametric uncertainties. The approach explicitly accounts for model uncertainty and ensures robust performance. By leveraging polynomial chaos theory (PCT) in conjunction with policy optimization techniques, the original stochastic system is lifted into a high-dimensional linear time-invariant (LTI) system with structured state-feedback control. A first-order gradient descent algorithm is then developed to directly optimize the structured feedback gain and iteratively minimize the LQR cost. We rigorously establish linear convergence of the gradient descent algorithm and show that the PCT-based approximation error decays algebraically at a rate $O(N^{-p})$ for any positive integer $p$, where $N$ denotes the order of the polynomials. Numerical examples demonstrate that the proposed method achieves significantly higher computational efficiency than conventional bilinear matrix inequality (BMI)-based approaches.

LQR for Systems with Probabilistic Parametric Uncertainties: A Gradient Method

Abstract

A gradient-based method is proposed for solving the linear quadratic regulator (LQR) problem for linear systems with nonlinear dependence on time-invariant probabilistic parametric uncertainties. The approach explicitly accounts for model uncertainty and ensures robust performance. By leveraging polynomial chaos theory (PCT) in conjunction with policy optimization techniques, the original stochastic system is lifted into a high-dimensional linear time-invariant (LTI) system with structured state-feedback control. A first-order gradient descent algorithm is then developed to directly optimize the structured feedback gain and iteratively minimize the LQR cost. We rigorously establish linear convergence of the gradient descent algorithm and show that the PCT-based approximation error decays algebraically at a rate for any positive integer , where denotes the order of the polynomials. Numerical examples demonstrate that the proposed method achieves significantly higher computational efficiency than conventional bilinear matrix inequality (BMI)-based approaches.

Paper Structure

This paper contains 19 sections, 29 theorems, 197 equations, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Preliminaries and Problem Statement
  3. Notations
  4. Preliminaries of Orthonormal Polynomials and Polynomial Chaos Expansion
  5. LQR Control for Systems with Probabilistic Parametric Uncertainty
  6. Policy Optimization Using Surrogate Model
  7. Finite-Dimensional Approximation Using PCT
  8. Gradient Computation for the Surrogate Objective Function
  9. Hessian Matrix Computation for the Surrogate Objective Function
  10. Theoretical Analysis of the Gradient Descent
  11. Convergence Rate of Gradient Descent
  12. Convergence of the PCE Approximation
  13. Numerical Studies
  14. Illustrative Example
  15. Mass-Spring System
  16. ...and 4 more sections

Key Result

Lemma 1

For any $K \in \hat{\pazocal{S}}_N$, the gradient of $\hat{\pazocal{J}}_N(K)$ is where $\pazocal{H}_N(K) = \pazocal{B}_N^\top {\pazocal{P}}_N(K)$.

Figures (5)

  • Figure 1: For the illustrative example, the evolution of the surrogate cost $\hat{\pazocal{J}}_N(K_k)$ and gradient $\nabla \hat{\pazocal{J}}_N(K_k)$ for fifth-order orthonormal polynomials.
  • Figure 2: For the illustrative example, the cost for the controller $K_r$ generated by the S-variable approach Ebihara2015s and the optimized controller $K_p$ by the PO algorithm in \ref{['eq:gradientDescent']}.
  • Figure 3: Four-mass-spring system.
  • Figure 4: For the mass-spring system, the evolution of the surrogate cost $\hat{\pazocal{J}}_N(K_k)$ and gradient $\nabla \hat{\pazocal{J}}_N(K_k)$ for the fifth-order of orthonormal polynomials.
  • Figure 5: For the mass-spring system, the cost for the controller $K_r$ generated by the S-variable approach Ebihara2015s and the optimized controller $K_p$ by the PO algorithm in \ref{['eq:gradientDescent']}.

Theorems & Definitions (54)

  • Definition 1
  • Remark 1
  • Lemma 1
  • proof
  • Lemma 2
  • proof
  • Lemma 3: Forward invariance of $\pazocal{S}(\bar{\xi},h)$
  • proof
  • Lemma 4: Perturbed PL inequality
  • proof
  • ...and 44 more