Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A framework for bilevel optimization that enables stochastic and global variance reduction algorithms

Mathieu Dagréou, Pierre Ablin, Samuel Vaiter, Thomas Moreau

TL;DR

The paper tackles stochastic bilevel optimization by introducing a joint dynamic framework where the inner minimizer and a linear-system solution evolve together with the outer variable. It defines unbiased directions $D_z$, $D_v$, and $D_x$ as simple sample-based sums, enabling single-loop updates and easy integration of variance-reduction techniques. The authors present SOBA, a single-sample SGD-like method, and SABA, a SAGA-like variance-reduction algorithm, and provide convergence guarantees: SOBA achieves $O(T^{-1/2})$ (or $O( rac{ ext{log}T}{\sqrt{T}})$ with decreasing steps) for the gradient norm, while SABA attains $O(N^{2/3}T^{-1})$ in general, and linear convergence under a PL inequality. Empirical results on hyperparameter selection and data cleaning show that SABA often outperforms existing stochastic bilevel methods, highlighting the practical impact of global variance reduction in bilevel settings.

Abstract

Bilevel optimization, the problem of minimizing a value function which involves the arg-minimum of another function, appears in many areas of machine learning. In a large scale empirical risk minimization setting where the number of samples is huge, it is crucial to develop stochastic methods, which only use a few samples at a time to progress. However, computing the gradient of the value function involves solving a linear system, which makes it difficult to derive unbiased stochastic estimates. To overcome this problem we introduce a novel framework, in which the solution of the inner problem, the solution of the linear system, and the main variable evolve at the same time. These directions are written as a sum, making it straightforward to derive unbiased estimates. The simplicity of our approach allows us to develop global variance reduction algorithms, where the dynamics of all variables is subject to variance reduction. We demonstrate that SABA, an adaptation of the celebrated SAGA algorithm in our framework, has $O(\frac1T)$ convergence rate, and that it achieves linear convergence under Polyak-Lojasciewicz assumption. This is the first stochastic algorithm for bilevel optimization that verifies either of these properties. Numerical experiments validate the usefulness of our method.

A framework for bilevel optimization that enables stochastic and global variance reduction algorithms

TL;DR

The paper tackles stochastic bilevel optimization by introducing a joint dynamic framework where the inner minimizer and a linear-system solution evolve together with the outer variable. It defines unbiased directions , , and as simple sample-based sums, enabling single-loop updates and easy integration of variance-reduction techniques. The authors present SOBA, a single-sample SGD-like method, and SABA, a SAGA-like variance-reduction algorithm, and provide convergence guarantees: SOBA achieves (or with decreasing steps) for the gradient norm, while SABA attains in general, and linear convergence under a PL inequality. Empirical results on hyperparameter selection and data cleaning show that SABA often outperforms existing stochastic bilevel methods, highlighting the practical impact of global variance reduction in bilevel settings.

Abstract

Bilevel optimization, the problem of minimizing a value function which involves the arg-minimum of another function, appears in many areas of machine learning. In a large scale empirical risk minimization setting where the number of samples is huge, it is crucial to develop stochastic methods, which only use a few samples at a time to progress. However, computing the gradient of the value function involves solving a linear system, which makes it difficult to derive unbiased stochastic estimates. To overcome this problem we introduce a novel framework, in which the solution of the inner problem, the solution of the linear system, and the main variable evolve at the same time. These directions are written as a sum, making it straightforward to derive unbiased estimates. The simplicity of our approach allows us to develop global variance reduction algorithms, where the dynamics of all variables is subject to variance reduction. We demonstrate that SABA, an adaptation of the celebrated SAGA algorithm in our framework, has convergence rate, and that it achieves linear convergence under Polyak-Lojasciewicz assumption. This is the first stochastic algorithm for bilevel optimization that verifies either of these properties. Numerical experiments validate the usefulness of our method.
Paper Structure (35 sections, 21 theorems, 204 equations, 6 figures, 1 table, 3 algorithms)

This paper contains 35 sections, 21 theorems, 204 equations, 6 figures, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. Proposed framework
  3. First example: the SOBA algorithm
  4. Global variance reduction with the SABA algorithm
  5. Theoretical analysis
  6. Background and assumptions
  7. Fundamental descent lemmas
  8. Analysis of SOBA
  9. SABA: a stochastic method with optimal rates
  10. Experiments
  11. Hyperparameters selection
  12. Data hyper-cleaning
  13. Conclusion
  14. Extensive comparison between stochastic methods for bilevel optimization
  15. Details on experiments
  16. ...and 20 more sections

Key Result

Proposition 2.1

Assume that for all $x\in\mathbb{R}^d$, $G(\cdot,x)$ is strongly convex. If $(z, v, x)$ is a zero of $(D_z, D_v, D_x)$, then $z = z^*(x)$, $v = v^*(x)$ and $\nabla h(x)=~0$.

Figures (6)

  • Figure 1: Convergence curves of the two proposed methods on a toy problem. SABA is a stochastic method that achieves fast convergence on the value function.
  • Figure 2: Comparison of SOBA and SABA with other stochastic bilevel optimization methods. For each algorithm, we plot the median performance over 10 runs. In both experiments, SABA achieves the best performance. The dashed lines are for one loop competitor methods, the dotted lines are for two loops methods and the solid lines are the proposed methods. Left: hyperparameter selection for $\ell^2$ penalized logistic regression on IJCNN1 dataset , Right: data hyper-cleaning on MNIST with $p=0.5$ corruption rate.
  • Figure : General framework
  • Figure B.1: Comparison of SOBA and SABA with other stochastic bilevel optimization methods in a problem of hyperparameter selection for $\ell^2$ penalized logistic regression on IJCNN1 dataset. For each algorithm, we plot the median performance over 10 runs. In both plots, SABA achieves the best performance. The dashed lines are for one loop competitor methods, the dotted lines are for two loops methods and the solid lines are the proposed methods. Left: performance in running time, Right: performance in number of gradient/Hessian-vector products sampled.
  • Figure B.2: Datacleaning experiment, with different corruption probability (higher means that more data are contamined). Top: Performance with respect to the number of gradient/Hessian-vector product sampled, Bottom: Performance with respect to running time
  • ...and 1 more figures

Theorems & Definitions (37)

  • Proposition 2.1
  • Lemma 3.4
  • Lemma 3.5
  • Lemma 3.9
  • Lemma 3.10
  • Theorem 1: Convergence of SOBA, fixed step size
  • Theorem 2: Convergence of SOBA, decreasing step size
  • Theorem 3: Convergence of SABA, smooth case
  • Theorem 4: Convergence of SABA, PL case
  • proof
  • ...and 27 more