Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Gray-Box Nonlinear Feedback Optimization

Zhiyu He, Saverio Bolognani, Michael Muehlebach, Florian Dörfler

TL;DR

The paper proposes gray-box feedback optimization that fuses approximate input-output sensitivities with model-free gradient estimates to drive nonlinear plants toward optimal steady states. By adaptively weighting model-based and model-free components, it provides performance guarantees and dynamic regret bounds for static and time-varying constrained problems. The approach enables efficient tracking of changing objectives while accommodating limited or imperfect sensitivity information, bridging model-based and model-free paradigms. The results illustrate improved sample efficiency and robustness, with practical extensions to running (time-varying) scenarios and clear guidelines for selecting combination coefficients based on sensitivity accuracy.

Abstract

Feedback optimization enables autonomous optimality seeking of a dynamical system through its closed-loop interconnection with iterative optimization algorithms. Among various iteration structures, model-based approaches require the input-output sensitivity of the system to construct gradients, whereas model-free approaches bypass this need by estimating gradients from real-time evaluations of the objective. These approaches own complementary benefits in sample efficiency and accuracy against model mismatch, i.e., errors of sensitivities. To achieve the best of both worlds, we propose gray-box feedback optimization controllers, featuring systematic incorporation of approximate sensitivities into model-free updates via adaptive convex combination. We quantify conditions on the accuracy of the sensitivities that render the gray-box approach preferable. We elucidate how the closed-loop performance is determined by the number of iterations, the problem dimension, and the cumulative effect of inaccurate sensitivities. The proposed controller contributes to a balanced closed-loop behavior, which retains provable sample efficiency and optimality guarantees for nonconvex problems. We further develop a running gray-box controller to handle constrained time-varying problems with changing objectives and steady-state maps.

Gray-Box Nonlinear Feedback Optimization

TL;DR

The paper proposes gray-box feedback optimization that fuses approximate input-output sensitivities with model-free gradient estimates to drive nonlinear plants toward optimal steady states. By adaptively weighting model-based and model-free components, it provides performance guarantees and dynamic regret bounds for static and time-varying constrained problems. The approach enables efficient tracking of changing objectives while accommodating limited or imperfect sensitivity information, bridging model-based and model-free paradigms. The results illustrate improved sample efficiency and robustness, with practical extensions to running (time-varying) scenarios and clear guidelines for selecting combination coefficients based on sensitivity accuracy.

Abstract

Feedback optimization enables autonomous optimality seeking of a dynamical system through its closed-loop interconnection with iterative optimization algorithms. Among various iteration structures, model-based approaches require the input-output sensitivity of the system to construct gradients, whereas model-free approaches bypass this need by estimating gradients from real-time evaluations of the objective. These approaches own complementary benefits in sample efficiency and accuracy against model mismatch, i.e., errors of sensitivities. To achieve the best of both worlds, we propose gray-box feedback optimization controllers, featuring systematic incorporation of approximate sensitivities into model-free updates via adaptive convex combination. We quantify conditions on the accuracy of the sensitivities that render the gray-box approach preferable. We elucidate how the closed-loop performance is determined by the number of iterations, the problem dimension, and the cumulative effect of inaccurate sensitivities. The proposed controller contributes to a balanced closed-loop behavior, which retains provable sample efficiency and optimality guarantees for nonconvex problems. We further develop a running gray-box controller to handle constrained time-varying problems with changing objectives and steady-state maps.
Paper Structure (29 sections, 13 theorems, 92 equations, 3 figures)

This paper contains 29 sections, 13 theorems, 92 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Motivations
  4. Contributions
  5. Organization
  6. Problem Formulation and Preliminaries
  7. Problem Formulation
  8. Preliminaries of Gradient Estimation
  9. Design of the Gray-Box Controller
  10. Gray-Box Controller
  11. Adaptive Combination Coefficients
  12. Performance Analysis and Comparison
  13. Performance Certificate
  14. Comparison with Model-based and Model-Free Controllers
  15. Constrained and Time-Varying Feedback Optimization
  16. ...and 14 more sections

Key Result

Lemma 1

If $\xi: \mathbb{R}^p \to \mathbb{R}$ is $L_{\xi}$-smooth, then $\xi_{\delta}(w)$ defined in eq:smooth_approx is also $L_{\xi}$-smooth, and where $w\in \mathbb{R}^p$, $\delta > 0$. If $\xi$ is convex, then $\xi_{\delta}$ is also convex.

Figures (3)

  • Figure 1: Interconnection of a physical plant \ref{['eq:sys_map']} and the gray-box feedback optimization controller \ref{['eq:hybrid_controller']}.
  • Figure 2: Comparison of our gray-box feedback optimization controller \ref{['eq:hybrid_controller']} and its counterparts to solve problem \ref{['eq:opt_problem_simulation']}. In the legend "FO" and "SL" stand for "feedback optimization" and "sensitivity learning", respectively.
  • Figure 3: Comparison of different controllers when interconnected with the system \ref{['eq:sys_simulation']} to solve the time-varying problem \ref{['eq:opt_problem_simulation_TV']}.

Theorems & Definitions (27)

  • Remark 1
  • Remark 2
  • Lemma 1: gao2018information
  • Lemma 2
  • proof
  • Lemma 3
  • proof
  • Theorem 4
  • proof
  • Theorem 5
  • ...and 17 more