Robust and structure exploiting optimization algorithms: An integral quadratic constraint approach

TL;DR

This paper addresses the design and analysis of gradient-based discrete-time optimization algorithms for strongly convex objectives with Lipschitz gradients, i.e., $f$ with parameter $\mu>0$ and gradient Lipschitz constant $L$. It adopts a robust control framework based on integral quadratic constraints (IQCs) to model the algorithm–gradient interaction as a feedback interconnection and to derive convergence guarantees via IQC-based lemmas and multiplier parametrizations including Zames-Falb multipliers. The authors also develop a synthesis path to construct robust, structure-exploiting algorithms with prespecified performance. The framework unifies analysis of existing methods and enables practical design of reliable optimization routines for control and signal processing applications.

Abstract

We consider the problem of analyzing and designing gradient-based discrete-time optimization algorithms for a class of unconstrained optimization problems having strongly convex objective functions with Lipschitz continuous gradient. By formulating the problem as a robustness analysis problem and making use of a suitable adaptation of the theory of integral quadratic constraints, we establish a framework that allows to analyze convergence rates and robustness properties of existing algorithms and enables the design of novel robust optimization algorithms with prespecified guarantees capable of exploiting additional structure in the objective function.