Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A new framework for constrained optimization via feedback control of Lagrange multipliers

V. Cerone, S. M. Fosson, S. Pirrera, D. Regruto

TL;DR

This work develops a continuous-time control-theoretic framework for equality-constrained optimization by treating the Lagrange multipliers as the control input and the constraint outputs as regulation targets. It introduces two concrete instantiations, PI-CMO and FL-CMO, and provides rigorous convergence analyses: global exponential convergence for PI-CMO under strong convexity and affine constraints, plus local (and global for strong convexity) convergence results for FL-CMO. The authors validate the methods through multiple numerical experiments, including convex quadratic problems, the Shidoku puzzle, gray-box system identification, and industrial process optimization, showing competitive or superior performance to state-of-the-art CT methods and traditional solvers. The results demonstrate the practical potential of a control-centric lens on constrained optimization, offering scalable first-order CT algorithms with favorable convergence properties and robustness characteristics.

Abstract

The continuous-time analysis of existing iterative algorithms for optimization has a long history. This work proposes a novel continuous-time control-theoretic framework for equality-constrained optimization. The key idea is to design a feedback control system where the Lagrange multipliers are the control input, and the output represents the constraints. The system converges to a stationary point of the constrained optimization problem through suitable regulation. Regarding the Lagrange multipliers, we consider two control laws: proportional-integral control and feedback linearization. These choices give rise to a family of different methods. We rigorously develop the related algorithms, theoretically analyze their convergence and present several numerical experiments to support their effectiveness concerning the state-of-the-art approaches.

A new framework for constrained optimization via feedback control of Lagrange multipliers

TL;DR

This work develops a continuous-time control-theoretic framework for equality-constrained optimization by treating the Lagrange multipliers as the control input and the constraint outputs as regulation targets. It introduces two concrete instantiations, PI-CMO and FL-CMO, and provides rigorous convergence analyses: global exponential convergence for PI-CMO under strong convexity and affine constraints, plus local (and global for strong convexity) convergence results for FL-CMO. The authors validate the methods through multiple numerical experiments, including convex quadratic problems, the Shidoku puzzle, gray-box system identification, and industrial process optimization, showing competitive or superior performance to state-of-the-art CT methods and traditional solvers. The results demonstrate the practical potential of a control-centric lens on constrained optimization, offering scalable first-order CT algorithms with favorable convergence properties and robustness characteristics.

Abstract

The continuous-time analysis of existing iterative algorithms for optimization has a long history. This work proposes a novel continuous-time control-theoretic framework for equality-constrained optimization. The key idea is to design a feedback control system where the Lagrange multipliers are the control input, and the output represents the constraints. The system converges to a stationary point of the constrained optimization problem through suitable regulation. Regarding the Lagrange multipliers, we consider two control laws: proportional-integral control and feedback linearization. These choices give rise to a family of different methods. We rigorously develop the related algorithms, theoretically analyze their convergence and present several numerical experiments to support their effectiveness concerning the state-of-the-art approaches.
Paper Structure (20 sections, 8 theorems, 117 equations, 12 figures, 1 table)

This paper contains 20 sections, 8 theorems, 117 equations, 12 figures, 1 table.

Table of Contents

  1. Introduction
  2. Problem statement and proposed framework
  3. Proposed framework: feedback control of Lagrange multipliers
  4. Related work
  5. First method: PI-CMO
  6. Global exponential convergence of PI-CMO for strongly convex problems
  7. Illustrative example
  8. PI-CMO in non-convex quadratic optimization with linear constraints
  9. Second method: FL-CMO
  10. Feedback linearization basics
  11. Non-interacting control of Lagrange multipliers
  12. Local convergence of FL-CMO
  13. Global exponential convergence of FL-CMO for strongly convex problems
  14. Numerical results
  15. PI-CMO and FL-CMO vs PDGD in convex optimization
  16. ...and 5 more sections

Key Result

Theorem 1

Let $x^{\star} \in \mathbb{R}^n$ be a local minimum of $f$ such that $h(x^{\star})=0$. Assume that $x^{\star}$ is regular, i.e., $\nabla h_1(x^{\star}), \dots,\nabla h_m(x^{\star})$ are linearly independent. Then, there exists a unique $\lambda^{\star} \in \mathbb{R}^m$ such that $(x^{\star},\lambda where $J_h(x)\in\mathbb{R}^{m,n}$ is the Jacobian matrix of $h$ evaluated in $x$.

Figures (12)

  • Figure 1: Structure of the proposed feedback control approach. The state and the output of $\mathcal{P}$ defined in \ref{['uncontrolled']} are fed back to the controller $\mathcal{K}$, whose output is the vector of the Lagrange multipliers.
  • Figure 2: First method: PI-CMO. We feedback the output $y(t)$ to the controller $\mathcal{K}$, which applies proportional an integral actions.
  • Figure 3: Second method: feedback linearization. We feedback the output $y(t)$ to the controller $\mathcal{K}$ to compute $v(t)$ according to \ref{['linear_control']}, and the state $x(t)$ to compute $u(t)$ according to \ref{['uabx']}.
  • Figure 4: Number of iterations to converge for PI-CMO, FL-CMO, and PDGD in a convex quadratic example, with $m$ constraints. The results are averaged over 400 random runs.
  • Figure 5: Shidoku puzzle to be solved.
  • ...and 7 more figures

Theorems & Definitions (28)

  • Theorem 1: First-order necessary conditions
  • Lemma 1
  • proof
  • Remark 1
  • Remark 2
  • Theorem 2: Global exponential convergence of PI-CMO
  • proof
  • Remark 3
  • Corollary 1
  • proof
  • ...and 18 more