Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Data-driven Estimation of the Algebraic Riccati Equation for the Discrete-Time Inverse Linear Quadratic Regulator Problem

Shuhei Sugiura, Ryo Ariizumi, Masaya Tanemura, Toru Asai, Shun-ichi Azuma

TL;DR

This work addresses estimating the algebraic Riccati equation (ARE) for a discrete-time ILQR problem when system matrices are unknown. It develops a data-driven approach that transforms ARE conditions into a linear equation in $P$, $Q$, and $R$ using state–input observations, proving equivalence to the ARE under appropriate data and rank assumptions. By exploiting prior information about $Q$ and $R$, the method achieves data economies, often requiring fewer observations than full system identification. Numerical experiments demonstrate accurate ARE recovery and robustness to noise, with improved data efficiency when prior structure is known. The approach offers a practical route for inverse optimal control in biological and other domains where models are hard to identify.

Abstract

In this paper, we propose a method for estimating the algebraic Riccati equation (ARE) with respect to an unknown discrete-time system from the system state and input observation. The inverse optimal control (IOC) problem asks, ``What objective function is optimized by a given control system?'' The inverse linear quadratic regulator (ILQR) problem is an IOC problem that assumes a linear system and quadratic objective function. The ILQR problem can be solved by solving a linear matrix inequality that contains the ARE. However, the system model is required to obtain the ARE, and it is often unknown in fields in which the IOC problem occurs, for example, biological system analysis. Our method directly estimates the ARE from the observation data without identifying the system. This feature enables us to economize the observation data using prior information about the objective function. We provide a data condition that is sufficient for our method to estimate the ARE. We conducted a numerical experiment to demonstrate that our method can estimate the ARE with less data than system identification if the prior information is sufficient.

Data-driven Estimation of the Algebraic Riccati Equation for the Discrete-Time Inverse Linear Quadratic Regulator Problem

TL;DR

This work addresses estimating the algebraic Riccati equation (ARE) for a discrete-time ILQR problem when system matrices are unknown. It develops a data-driven approach that transforms ARE conditions into a linear equation in , , and using state–input observations, proving equivalence to the ARE under appropriate data and rank assumptions. By exploiting prior information about and , the method achieves data economies, often requiring fewer observations than full system identification. Numerical experiments demonstrate accurate ARE recovery and robustness to noise, with improved data efficiency when prior structure is known. The approach offers a practical route for inverse optimal control in biological and other domains where models are hard to identify.

Abstract

In this paper, we propose a method for estimating the algebraic Riccati equation (ARE) with respect to an unknown discrete-time system from the system state and input observation. The inverse optimal control (IOC) problem asks, ``What objective function is optimized by a given control system?'' The inverse linear quadratic regulator (ILQR) problem is an IOC problem that assumes a linear system and quadratic objective function. The ILQR problem can be solved by solving a linear matrix inequality that contains the ARE. However, the system model is required to obtain the ARE, and it is often unknown in fields in which the IOC problem occurs, for example, biological system analysis. Our method directly estimates the ARE from the observation data without identifying the system. This feature enables us to economize the observation data using prior information about the objective function. We provide a data condition that is sufficient for our method to estimate the ARE. We conducted a numerical experiment to demonstrate that our method can estimate the ARE with less data than system identification if the prior information is sufficient.
Paper Structure (9 sections, 41 equations, 6 figures)

This paper contains 9 sections, 41 equations, 6 figures.

Table of Contents

  1. INTRODUCTION
  2. PROBLEM FORMULATION
  3. DATA-DRIVEN ESTIMATION OF ARE
  4. NUMERICAL EXPERIMENT
  5. Distance Between Solution Spaces
  6. Experiment 1
  7. Experiment 2
  8. Experiment 3
  9. CONCLUSIONS

Figures (6)

  • Figure 1: Ratio $\frac{N_{d\rm min}}{n+m}$ when $5\le n\le 200$ and $5\le m\le 200$. The contour lines are not smooth, but this is not a mistake.
  • Figure 2: Singular values of coefficient matrices $\Theta$ and $\hat{\Theta}$ in Experiment 1.
  • Figure 3: Relationship between distance $d{\left(S,\hat{S}\right)}$ and number of significant digits in the computation in Experiment 1.
  • Figure 4: Singular values of coefficient matrices $\Theta$ and $\hat{\Theta}$ in Experiment 2.
  • Figure 5: Singular values of coefficient matrices $\Theta$ and $\hat{\Theta}$ in Experiment 3.
  • ...and 1 more figures

Theorems & Definitions (4)

  • proof
  • proof
  • proof
  • proof