Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Convergence Analysis for A Stochastic Maximum Principle Based Data Driven Feedback Control Algorithm

Siming Liang, Hui Sun, Richard Archibald, Feng Bao

TL;DR

This work analyzes the convergence of a data-driven online feedback control algorithm that couples particle-filter state estimation from partial, noisy observations with a stochastic maximum principle–based SGD solver. By integrating established particle-filter weak convergence results with stochastic-gradient convergence analyses for SMP solvers, it derives a weak convergence result for the data-driven optimization procedure and provides a rigorously justified framework for online control in high-dimensional, partially observed settings. Two numerical experiments—the 4-D linear-quadratic problem with nonlinear observations and a 2D Dubins vehicle maneuvering problem—validate the theory, showing that the state distribution and the computed control converge as the particle count and iteration depth increase. The methodology offers a practical, scalable path to reliable online data-driven feedback control where exact state information is unavailable.

Abstract

This paper presents convergence analysis of a novel data-driven feedback control algorithm designed for generating online controls based on partial noisy observational data. The algorithm comprises a particle filter-enabled state estimation component, estimating the controlled system's state via indirect observations, alongside an efficient stochastic maximum principle type optimal control solver. By integrating weak convergence techniques for the particle filter with convergence analysis for the stochastic maximum principle control solver, we derive a weak convergence result for the optimization procedure in search of optimal data-driven feedback control. Numerical experiments are performed to validate the theoretical findings.

Convergence Analysis for A Stochastic Maximum Principle Based Data Driven Feedback Control Algorithm

TL;DR

This work analyzes the convergence of a data-driven online feedback control algorithm that couples particle-filter state estimation from partial, noisy observations with a stochastic maximum principle–based SGD solver. By integrating established particle-filter weak convergence results with stochastic-gradient convergence analyses for SMP solvers, it derives a weak convergence result for the data-driven optimization procedure and provides a rigorously justified framework for online control in high-dimensional, partially observed settings. Two numerical experiments—the 4-D linear-quadratic problem with nonlinear observations and a 2D Dubins vehicle maneuvering problem—validate the theory, showing that the state distribution and the computed control converge as the particle count and iteration depth increase. The methodology offers a practical, scalable path to reliable online data-driven feedback control where exact state information is unavailable.

Abstract

This paper presents convergence analysis of a novel data-driven feedback control algorithm designed for generating online controls based on partial noisy observational data. The algorithm comprises a particle filter-enabled state estimation component, estimating the controlled system's state via indirect observations, alongside an efficient stochastic maximum principle type optimal control solver. By integrating weak convergence techniques for the particle filter with convergence analysis for the stochastic maximum principle control solver, we derive a weak convergence result for the optimization procedure in search of optimal data-driven feedback control. Numerical experiments are performed to validate the theoretical findings.
Paper Structure (16 sections, 6 theorems, 121 equations, 7 figures, 1 algorithm)

This paper contains 16 sections, 6 theorems, 121 equations, 7 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. An efficient algorithm for data driven feedback control
  3. Problem setting for the data driven optimal control problem.
  4. The algorithm for solving the data driven optimal control problem.
  5. The optimization procedure for optimal control
  6. Numerical Approach for Data Driven Feedback Control by PF-SGD
  7. Convergence analysis
  8. Notations and assumptions
  9. The convergence theorem for the data driven feedback control algorithm
  10. Numerical example
  11. Example 1. Linear quadratic control problem with nonlinear observations
  12. Experimental design
  13. Performance experiment
  14. Convergence experiment
  15. Example 2. 2D Dubins vehicle maneuvering problem
  16. ...and 1 more sections

Key Result

Lemma 3.1

We assume that there exists $\kappa \in (0,1]$. The following is true:

Figures (7)

  • Figure 1: Estimated control vs True optimal control
  • Figure 2: Estimated state vs True state
  • Figure 3: Error VS Number of Particles
  • Figure 4: Error VS Number of Steps
  • Figure 5: Controlled trajectory from (0,0) to (1,1)
  • ...and 2 more figures

Theorems & Definitions (16)

  • Lemma 3.1
  • Remark
  • Remark
  • Lemma 3.2
  • proof
  • Remark
  • Lemma 3.3
  • proof
  • Lemma 3.4
  • proof
  • ...and 6 more