Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the Convex Interpolation for Linear Operators

Nizar Bousselmi, Zhicheng Deng, Jie Lu, Francois Glineur, Julien M. Hendrickx

TL;DR

The work addresses sharp worst-case analysis of optimization methods by developing interpolation conditions for linear operators under Gram representations. It reveals a fundamental limitation: convex Gram constraints can only characterize operator classes with spectra confined to a fixed subset, and then overcomes this for $\mathcal{L}_{[\mu,L]}$ and unions of spectral sets via polar decomposition. The framework is demonstrated through applications to Gradient Method and Chambolle-Pock, yielding new numerical worst-case guarantees within the Performance Estimation Problem. This provides a principled tool for spectral-structure-aware interpolation analyses and clarifies how eigenvalue/singular-value spectra govern worst-case behavior in convex optimization algorithms.

Abstract

The worst-case performance of an optimization method on a problem class can be analyzed using a finite description of the problem class, known as interpolation conditions. In this work, we study interpolation conditions for linear operators given scalar products between discrete inputs and outputs. First, we show that if only convex constraints on the scalar products of inputs and outputs are allowed,it is only possible to characterize classes of linear operators or symmetric linear operator whose all singular values or eigenvalues belong to some subset of R. Then, we propose new interpolation conditions for linear operators with minimal and maximal singular values and linear operators whose eigenvalues or singular values belong to unions of subsets. Finally, we illustrate the new interpolation conditions through the analysis of the Gradient and Chambolle-Pock methods. It allows to obtain new numerical worst-case guarantees on these methods.

On the Convex Interpolation for Linear Operators

TL;DR

The work addresses sharp worst-case analysis of optimization methods by developing interpolation conditions for linear operators under Gram representations. It reveals a fundamental limitation: convex Gram constraints can only characterize operator classes with spectra confined to a fixed subset, and then overcomes this for and unions of spectral sets via polar decomposition. The framework is demonstrated through applications to Gradient Method and Chambolle-Pock, yielding new numerical worst-case guarantees within the Performance Estimation Problem. This provides a principled tool for spectral-structure-aware interpolation analyses and clarifies how eigenvalue/singular-value spectra govern worst-case behavior in convex optimization algorithms.

Abstract

The worst-case performance of an optimization method on a problem class can be analyzed using a finite description of the problem class, known as interpolation conditions. In this work, we study interpolation conditions for linear operators given scalar products between discrete inputs and outputs. First, we show that if only convex constraints on the scalar products of inputs and outputs are allowed,it is only possible to characterize classes of linear operators or symmetric linear operator whose all singular values or eigenvalues belong to some subset of R. Then, we propose new interpolation conditions for linear operators with minimal and maximal singular values and linear operators whose eigenvalues or singular values belong to unions of subsets. Finally, we illustrate the new interpolation conditions through the analysis of the Gradient and Chambolle-Pock methods. It allows to obtain new numerical worst-case guarantees on these methods.

Paper Structure

This paper contains 17 sections, 19 theorems, 52 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Worst-case analysis and Gram representation
  3. Contributions and outline of the paper
  4. Notations and conventions
  5. Limitation of representation of linear operators through Gram matrices
  6. Gram matrices associated with classes of linear operators
  7. Main result
  8. Set of Gram matrices and interpolation conditions
  9. Interpolation conditions for $\mathcal{L}_{[\mu,L]}$
  10. Interpolation conditions for union of eigenvalues and singular values
  11. Application to Performance Estimation and worst-case analysis
  12. Worst-case performance of Gradient Method on smooth convex functions
  13. Performance on linear operators with non-zero minimal singular value
  14. Performance on linear operators with eigenvalues on a union of intervals
  15. Chambolle-Pock Method for composite optimization problem
  16. ...and 2 more sections

Key Result

Theorem 1.1

Given $L\geq 0$. The sets of pairs $\{(x_i,y_i)\}_{i\in [N_1]}$ and $\{(u_j,v_j)\}_{j\in [N_2]}$ are $\mathcal{L}_{[0,L]}$-interpolable if, and only if,

Figures (1)

  • Figure 1: Comparison of the worst-case linear operators when allowing the whole interval (red curve) or only the union of two sub-intervals (blue curve).

Theorems & Definitions (40)

  • Definition 1.1: Operator interpolability
  • Theorem 1.1: $\mathcal{L}_{[0,L]}$-interpolation conditions, Theorem 3.1 of bousselmi2024interpolation
  • Theorem 1.2: $\mathcal{S}_{[\mu,L]}$-interpolation conditions, Theorem 3.3 of bousselmi2024interpolation
  • Definition 2.1: Gram matrix associated with a linear operator
  • Definition 2.2: Set of Gram matrices associated with a class of linear operator
  • Lemma 2.1: Unitary invariance of Gram matrices
  • proof
  • Definition 2.3: Spectrum of matrices
  • Corollary 2.2: Spectrum characterization
  • proof
  • ...and 30 more