Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A feedback control approach to convex optimization with inequality constraints

V. Cerone, S. M. Fosson, S. Pirrera, D. Regruto

TL;DR

The paper develops a continuous-time PI-control-inspired algorithm for smooth, strongly convex optimization with inequality constraints, built on an augmented Lagrangian and a state-feedback term that augments the Lagrange multiplier dynamics. It provides a global exponential convergence proof under standard spectral bounds and demonstrates faster convergence than primal-dual gradient dynamics in numerical experiments on quadratic programming and system identification. The approach unifies control-theoretic intuition with convex optimization, offering practical speedups and simpler analysis. Future work aims to relax key assumptions such as affinity and smoothness to broaden applicability.

Abstract

We propose a novel continuous-time algorithm for inequality-constrained convex optimization inspired by proportional-integral control. Unlike the popular primal-dual gradient dynamics, our method includes a proportional term to control the primal variable through the Lagrange multipliers. This approach has both theoretical and practical advantages. On the one hand, it simplifies the proof of the exponential convergence in the case of smooth, strongly convex problems, with a more straightforward assessment of the convergence rate concerning prior literature. On the other hand, through several examples, we show that the proposed algorithm converges faster than primal-dual gradient dynamics. This paper aims to illustrate these points by thoroughly analyzing the algorithm convergence and discussing some numerical simulations.

A feedback control approach to convex optimization with inequality constraints

TL;DR

The paper develops a continuous-time PI-control-inspired algorithm for smooth, strongly convex optimization with inequality constraints, built on an augmented Lagrangian and a state-feedback term that augments the Lagrange multiplier dynamics. It provides a global exponential convergence proof under standard spectral bounds and demonstrates faster convergence than primal-dual gradient dynamics in numerical experiments on quadratic programming and system identification. The approach unifies control-theoretic intuition with convex optimization, offering practical speedups and simpler analysis. Future work aims to relax key assumptions such as affinity and smoothness to broaden applicability.

Abstract

We propose a novel continuous-time algorithm for inequality-constrained convex optimization inspired by proportional-integral control. Unlike the popular primal-dual gradient dynamics, our method includes a proportional term to control the primal variable through the Lagrange multipliers. This approach has both theoretical and practical advantages. On the one hand, it simplifies the proof of the exponential convergence in the case of smooth, strongly convex problems, with a more straightforward assessment of the convergence rate concerning prior literature. On the other hand, through several examples, we show that the proposed algorithm converges faster than primal-dual gradient dynamics. This paper aims to illustrate these points by thoroughly analyzing the algorithm convergence and discussing some numerical simulations.
Paper Structure (9 sections, 3 theorems, 49 equations, 3 figures, 2 tables)

This paper contains 9 sections, 3 theorems, 49 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Problem statement and related work
  3. Proposed approach
  4. Convergence analysis
  5. Example on the convergence rate of PI and PDGD
  6. Numerical results
  7. Example 1: Quadratic programming
  8. Example 2: Linear system identification
  9. Conclusion

Key Result

Theorem 1

Let $x^{\star} \in \mathbb{R}^n$ be a minimum of $f$ subject to the constraints $h(x)\leq 0$. Then, there exists a unique $\lambda^{\star} \in \mathbb{R}^m$ such that

Figures (3)

  • Figure 1: Example 1: constraints violation.
  • Figure 2: Example 1: distance from the optimum.
  • Figure 3: Example 2: validation of identified models

Theorems & Definitions (8)

  • Theorem 1: KKT first-order conditions
  • Proposition 1
  • proof
  • Theorem 2: Global exponential convergence
  • proof
  • Remark 1
  • Remark 2
  • Remark 3