Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Data-Driven Algorithm for Model-Free Control Synthesis

Sean Bowerfind, Matthew R. Kirchner, Gary Hewer

TL;DR

This work develops a data-driven framework for synthesizing the infinite-horizon LQR controller for continuous-time systems without explicit dynamics models. It exploits a necessary condition on the value function, discretizes it, and formulates an implicit NLP to recover the LQR gain $K$ (and a feedforward $F$ for tracking) from finite data, robust to noise via a PSD factorization $P=L^TL$. The method extends to reference tracking and to mixed-model scenarios where part of the dynamics is known, solving for $(K,F)$ or $(K,F)$ with the known dynamics in the loop. Demonstrations include a known linear-system benchmark and a real-world flight-test on a subscale UAV, showing model-free controllers closely match traditional LQR performance and effectively track references under realistic conditions. The approach offers a practical route to LQR-like control when modeling is difficult or impractical, with potential for real-time adaptive extensions and data-driven validation in aerospace and other nonlinear domains.

Abstract

Presented is an algorithm to synthesize the optimal infinite-horizon LQR feedback controller for continuous-time systems. The algorithm does not require knowledge of the system dynamics but instead uses only a finite-length sampling of arbitrary input-output data. The algorithm is based on a constrained optimization problem that enforces a necessary condition on the dynamics of the optimal value function along any trajectory. In addition to calculating the standard LQR gain matrix, a feedforward gain can be found to implement a reference tracking controller. This paper presents a theoretical justification for the method and shows several examples, including a validation test on a real scale aircraft.

A Data-Driven Algorithm for Model-Free Control Synthesis

TL;DR

This work develops a data-driven framework for synthesizing the infinite-horizon LQR controller for continuous-time systems without explicit dynamics models. It exploits a necessary condition on the value function, discretizes it, and formulates an implicit NLP to recover the LQR gain (and a feedforward for tracking) from finite data, robust to noise via a PSD factorization . The method extends to reference tracking and to mixed-model scenarios where part of the dynamics is known, solving for or with the known dynamics in the loop. Demonstrations include a known linear-system benchmark and a real-world flight-test on a subscale UAV, showing model-free controllers closely match traditional LQR performance and effectively track references under realistic conditions. The approach offers a practical route to LQR-like control when modeling is difficult or impractical, with potential for real-time adaptive extensions and data-driven validation in aerospace and other nonlinear domains.

Abstract

Presented is an algorithm to synthesize the optimal infinite-horizon LQR feedback controller for continuous-time systems. The algorithm does not require knowledge of the system dynamics but instead uses only a finite-length sampling of arbitrary input-output data. The algorithm is based on a constrained optimization problem that enforces a necessary condition on the dynamics of the optimal value function along any trajectory. In addition to calculating the standard LQR gain matrix, a feedforward gain can be found to implement a reference tracking controller. This paper presents a theoretical justification for the method and shows several examples, including a validation test on a real scale aircraft.
Paper Structure (22 sections, 1 theorem, 65 equations, 10 figures, 4 tables)

This paper contains 22 sections, 1 theorem, 65 equations, 10 figures, 4 tables.

Table of Contents

  1. Introduction
  2. LQR Optimal Control Fundamentals
  3. Model-Free Necessary Conditions for Optimality
  4. Data-Driven Control Synthesis
  5. A Note On Data Collection
  6. A Note on Non-Zero Equilibrium Points
  7. Model-Free Reference Tracking Control
  8. Mixed-Model Controller Design
  9. Mixed-Model with Reference Tracking
  10. Results
  11. Known Linear System
  12. Model-Free Controller Flight Test Program
  13. Test Aircraft Specifications
  14. Flight Computer and Sensor Package
  15. Aircraft Dynamics Fundamentals
  16. ...and 7 more sections

Key Result

Theorem 1

Suppose the trajectory pair $\left[0,T\right]\ni t \mapsto\left\{ \hat{x}(t), \hat{u}(t)\right\}\in\mathbb{R}^n\times\mathbb{R}^m$, is a solution to $\left( eq:LTIsys \right)$ in the Carathéodory sense, $P=P^\top \succeq 0$ is the solution to the Algebraic Riccati Equation $\left( eq:ARE \right)$, a almost everywhere on $t\in [0,T]$.

Figures (10)

  • Figure 1: Solving the reference tracking NLP \ref{['eq:NLP_ref']} recovers the LQR feedback gain $K$ and feedforward gain $F$, both shown in blue. $H$ is a given matrix that defines the tracking such that $y=r$. The reference tracking controller is implemented by using the controller $u=-K\left(x-Hr\right)+Fr$.
  • Figure 2: The testing paradigm for Example \ref{['subsec: known_linear_results']}
  • Figure 3: The closed-loop simulation of A-4D lateral dynamics subject to roll angle tracking commands. The true LQR solution with $K^*$ is shown with the dashed red line. The model-free controller response using $\hat{K}$ is shown with the solid blue line. Roll angle tracking commands are indicated with the dashed black line.
  • Figure 4: The closed-loop simulation of A-4D lateral dynamics with additional control surface actuator dynamics. The ground truth, generated by $\hat{K}$, is shown with the dashed red line. The mixed-model controller $K^*$ produces the response shown with the solid blue line. The desired roll angle tracking commands are shown with the dashed black line.
  • Figure 5: The Freewing AL37 test aircraft.
  • ...and 5 more figures

Theorems & Definitions (1)

  • Theorem 1