Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Stochastic models for online optimization

Umberto Casti, Sandro Zampieri

TL;DR

This work develops control-theoretic online optimization methods for time-varying quadratic costs driven by stochastic linear dynamics. It introduces a Kalman-filter-inspired estimator and an $\mathcal{H}_\infty$-robust controller to track the moving minimizer $\mathbf{c}_k$ using gradient observations, with a formal objective $J$ related to the error $\mathbf{e}_k = \mathbf{x}_k-\mathbf{c}_k$ and an $\mathcal{H}_2$/$\mathcal{H}_\infty$-norm framework. The methods address both stable and unstable dynamics via $\,\mu$-synthesis and, in the unstable case, a precompensated design to handle uncontrollable poles, with simulations showing superior performance over online gradient descent and a deterministic baseline. The results indicate improved tracking robustness under noise, model uncertainty, and potential instability, highlighting practical impact for dynamic, noisy optimization in control and signal processing applications.

Abstract

In this paper, we propose control-theoretic methods as tools for the design of online optimization algorithms that are able to address dynamic, noisy, and partially uncertain time-varying quadratic objective functions. Our approach introduces two algorithms specifically tailored for scenarios where the cost function follows a stochastic linear model. The first algorithm is based on a Kalman filter-inspired approach, leveraging state estimation techniques to account for the presence of noise in the evolution of the objective function. The second algorithm applies $\mathcal{H}_\infty$-robust control strategies to enhance performance under uncertainty, particularly in cases in which model parameters are characterized by a high variability. Through numerical experiments, we demonstrate that our algorithms offer significant performance advantages over the traditional gradient-based method and also over the optimization strategy proposed in arXiv:2205.13932 based on deterministic models.

Stochastic models for online optimization

TL;DR

This work develops control-theoretic online optimization methods for time-varying quadratic costs driven by stochastic linear dynamics. It introduces a Kalman-filter-inspired estimator and an -robust controller to track the moving minimizer using gradient observations, with a formal objective related to the error and an /-norm framework. The methods address both stable and unstable dynamics via -synthesis and, in the unstable case, a precompensated design to handle uncontrollable poles, with simulations showing superior performance over online gradient descent and a deterministic baseline. The results indicate improved tracking robustness under noise, model uncertainty, and potential instability, highlighting practical impact for dynamic, noisy optimization in control and signal processing applications.

Abstract

In this paper, we propose control-theoretic methods as tools for the design of online optimization algorithms that are able to address dynamic, noisy, and partially uncertain time-varying quadratic objective functions. Our approach introduces two algorithms specifically tailored for scenarios where the cost function follows a stochastic linear model. The first algorithm is based on a Kalman filter-inspired approach, leveraging state estimation techniques to account for the presence of noise in the evolution of the objective function. The second algorithm applies -robust control strategies to enhance performance under uncertainty, particularly in cases in which model parameters are characterized by a high variability. Through numerical experiments, we demonstrate that our algorithms offer significant performance advantages over the traditional gradient-based method and also over the optimization strategy proposed in arXiv:2205.13932 based on deterministic models.

Paper Structure

This paper contains 11 sections, 39 equations, 11 figures.

Table of Contents

  1. Introduction
  2. Problem formulation and background
  3. Algorithms
  4. Kalman-inspired algorithm
  5. Robust $\mathcal{H}_\infty$-inspired algorithm
  6. ${\mathbold{F}}$ stable
  7. ${\mathbold{F}}$ unstable
  8. Simulations
  9. ${\mathbold{F}}$ stable
  10. ${\mathbold{F}}$ unstable
  11. Conclusions

Figures (11)

  • Figure 1: Proposed control scheme for the solution of \ref{['eq:general-online-optimization']}.
  • Figure 2: Control scheme of Fig. \ref{['fig:block-diagram']} when ${\mathbold{C}}(z) = c(z) {\mathbold{I}}_n$ and condition \ref{['eq:gradE']} holds.
  • Figure 3: Block scheme representation of the transfer function \ref{['eq:scalarTF']}.
  • Figure 4: General framework for $\mu$-synthesis.
  • Figure 5: General framework shown in Fig. \ref{['fig:block-diagram-generalFramework']}, tailored to the specific control scheme depicted in Fig. \ref{['fig:block-diagram-caseScalar']}.
  • ...and 6 more figures

Theorems & Definitions (3)

  • Remark 1
  • Remark 2
  • Remark 3