Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PEPit: computer-assisted worst-case analyses of first-order optimization methods in Python

Baptiste Goujaud, Céline Moucer, François Glineur, Julien Hendrickx, Adrien Taylor, Aymeric Dieuleveut

Abstract

PEPit is a Python package aiming at simplifying the access to worst-case analyses of a large family of first-order optimization methods possibly involving gradient, projection, proximal, or linear optimization oracles, along with their approximate, or Bregman variants. In short, PEPit is a package enabling computer-assisted worst-case analyses of first-order optimization methods. The key underlying idea is to cast the problem of performing a worst-case analysis, often referred to as a performance estimation problem (PEP), as a semidefinite program (SDP) which can be solved numerically. To do that, the package users are only required to write first-order methods nearly as they would have implemented them. The package then takes care of the SDP modeling parts, and the worst-case analysis is performed numerically via a standard solver.

PEPit: computer-assisted worst-case analyses of first-order optimization methods in Python

Abstract

PEPit is a Python package aiming at simplifying the access to worst-case analyses of a large family of first-order optimization methods possibly involving gradient, projection, proximal, or linear optimization oracles, along with their approximate, or Bregman variants. In short, PEPit is a package enabling computer-assisted worst-case analyses of first-order optimization methods. The key underlying idea is to cast the problem of performing a worst-case analysis, often referred to as a performance estimation problem (PEP), as a semidefinite program (SDP) which can be solved numerically. To do that, the package users are only required to write first-order methods nearly as they would have implemented them. The package then takes care of the SDP modeling parts, and the worst-case analysis is performed numerically via a standard solver.
Paper Structure (18 sections, 1 theorem, 12 equations, 1 figure)

This paper contains 18 sections, 1 theorem, 12 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Organization of the paper.
  3. Related works.
  4. Dependencies.
  5. PEPit on a simple example
  6. A performance estimation problem for the gradient method
  7. Code
  8. Imports.
  9. Initialization of PEPit.
  10. Specifying the problem class.
  11. Algorithm initialization.
  12. Algorithm implementation.
  13. Setting up a performance measure.
  14. Solving the PEP.
  15. Output.
  16. ...and 3 more sections

Key Result

theorem 1

taylor2017smooth Let $I$ be an index set and $S=\{(x_i,g_i,f_i)\}_{i\in I}$ be such that $x_i,g_i\in\mathbb{R}^d$ and $f_i\in\mathbb{R}$ for all $i\in I$. There exists a function $F\in\mathcal{F}_{\mu,L}(\mathbb{R}^d)$ such that $f_i=F(x_i)$ and $g_i=\nabla F(x_i)$ (for all $i\in I$) if and only if

Figures (1)

  • Figure 1: Comparison: worst-case guarantee from PEPit (plain blue) and theoretical tight worst-case bound \ref{['eq:tau_GD_ex']} for gradient descent in terms of $\frac{\|x_n-y_n\|^2_2}{\|x_0-y_0\|^2_2}$ (dashed red). Problem parameters fixed to $\mu=0.1$ and $L=1$.

Theorems & Definitions (2)

  • theorem 1
  • remark thmcounterremark: Important ingredients for the SDP reformulations