Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A stochastic gradient method for trilevel optimization

Tommaso Giovannelli, Griffin Dean Kent, Luis Nunes Vicente

TL;DR

The paper introduces Trilevel Stochastic Gradient (TSG), the first stochastic first-order method for unconstrained trilevel optimization, and provides convergence guarantees that tolerate inexact adjoint gradients and noisy derivative estimates. It derives the adjoint-gradient framework for the trilevel problem, analyzes stability under a comprehensive set of assumptions (including strong convexity at lower levels and unbiased, bounded-variance stochastic estimates), and proves a sublinear convergence rate in the nonconvex UL setting. Practically, it offers two scalable variants, TSG-N-FD and TSG-AD, to approximate implicit derivatives without resorting to costly third-order tensors, and demonstrates their performance on synthetic trilevel problems and an adversarial hyperparameter-tuning task. The results indicate that TSG-AD is generally more robust to noise and complexity than TSG-N-FD, highlighting the method's potential for scalable, hierarchical ML formulations while also outlining future work to relax strong convexity assumptions and extend applicability. Overall, the work provides a foundational, scalable approach to stochastic trilevel optimization with rigorous theory and practical algorithms.

Abstract

With the success that the field of bilevel optimization has seen in recent years, similar methodologies have started being applied to solving more difficult applications that arise in trilevel optimization. At the helm of these applications are new machine learning formulations that have been proposed in the trilevel context and, as a result, efficient and theoretically sound stochastic methods are required. In this work, we propose the first-ever stochastic gradient descent method for solving unconstrained trilevel optimization problems and provide a convergence theory that covers all forms of inexactness of the trilevel adjoint gradient, such as the inexact solutions of the middle-level and lower-level problems, inexact computation of the trilevel adjoint formula, and noisy estimates of the gradients, Hessians, Jacobians, and tensors of third-order derivatives involved. We also demonstrate the promise of our approach by providing numerical results on both synthetic trilevel problems and trilevel formulations for hyperparameter adversarial tuning.

A stochastic gradient method for trilevel optimization

TL;DR

The paper introduces Trilevel Stochastic Gradient (TSG), the first stochastic first-order method for unconstrained trilevel optimization, and provides convergence guarantees that tolerate inexact adjoint gradients and noisy derivative estimates. It derives the adjoint-gradient framework for the trilevel problem, analyzes stability under a comprehensive set of assumptions (including strong convexity at lower levels and unbiased, bounded-variance stochastic estimates), and proves a sublinear convergence rate in the nonconvex UL setting. Practically, it offers two scalable variants, TSG-N-FD and TSG-AD, to approximate implicit derivatives without resorting to costly third-order tensors, and demonstrates their performance on synthetic trilevel problems and an adversarial hyperparameter-tuning task. The results indicate that TSG-AD is generally more robust to noise and complexity than TSG-N-FD, highlighting the method's potential for scalable, hierarchical ML formulations while also outlining future work to relax strong convexity assumptions and extend applicability. Overall, the work provides a foundational, scalable approach to stochastic trilevel optimization with rigorous theory and practical algorithms.

Abstract

With the success that the field of bilevel optimization has seen in recent years, similar methodologies have started being applied to solving more difficult applications that arise in trilevel optimization. At the helm of these applications are new machine learning formulations that have been proposed in the trilevel context and, as a result, efficient and theoretically sound stochastic methods are required. In this work, we propose the first-ever stochastic gradient descent method for solving unconstrained trilevel optimization problems and provide a convergence theory that covers all forms of inexactness of the trilevel adjoint gradient, such as the inexact solutions of the middle-level and lower-level problems, inexact computation of the trilevel adjoint formula, and noisy estimates of the gradients, Hessians, Jacobians, and tensors of third-order derivatives involved. We also demonstrate the promise of our approach by providing numerical results on both synthetic trilevel problems and trilevel formulations for hyperparameter adversarial tuning.
Paper Structure (33 sections, 14 theorems, 206 equations, 12 figures, 3 tables, 7 algorithms)

This paper contains 33 sections, 14 theorems, 206 equations, 12 figures, 3 tables, 7 algorithms.

Table of Contents

  1. Introduction
  2. Trilevel optimization in the literature
  3. Contributions of this paper
  4. Trilevel optimization
  5. The trilevel stochastic gradient method
  6. Convergence analysis of the TSG method
  7. Assumptions on the trilevel problem
  8. Convergence of the TSG method
  9. Numerical experiments
  10. Our practical TSG methods
  11. Numerical results for synthetic trilevel problems
  12. Numerical results for trilevel adversarial hyperparameter tuning
  13. Conclusion
  14. Derivation of the trilevel adjoint gradient
  15. Discussion on the convergence analysis of the TSG method
  16. ...and 18 more sections

Key Result

Theorem 3.1

Under Assumptions as:tri_lip_cont--as:TSG_bounded_var, choose the step-sizes $\alpha_i = 1/\sqrt{I}$, $\beta_i = (1/\sqrt{J}) \alpha_i$, and $\gamma_i = (1/(\sqrt{J}\sqrt{K}))\alpha_i$. Then, the iterates $\{x^i\}_{i\geq0}$ generated by Algorithm alg:TSG satisfy $\frac{1}{I}\sum_{i=0}^{I-1} \mathbb{

Figures (12)

  • Figure 1: Quadratic problem, deterministic case.
  • Figure 2: Quartic problem, deterministic case.
  • Figure 3: Quadratic problem, stochastic case (low noise: two left plots; high noise: two right plots).
  • Figure 4: Quartic problem, stochastic case (low noise: two left plots; high noise: two right plots).
  • Figure 5: Adversarial learning formulation \ref{['prob:trilevel_adv_train']}, red wine quality dataset.
  • ...and 7 more figures

Theorems & Definitions (15)

  • Theorem 3.1: Convergence of TSG -- Nonconvex $f$
  • Remark 3.1
  • Proposition A.1: Trilevel adjoint gradient
  • Lemma B.1: Descent of the true trilevel UL problem
  • Lemma B.2: Error bounds of the trilevel LL problem
  • Lemma B.3: Auxiliary error bounds of the trilevel LL problem
  • Lemma B.4: Error bounds of the trilevel ML problem
  • Lemma D.1: Bounds on bias of $\nabla z$ and $\nabla^2\Bar{f}$
  • Lemma D.2: Bounds on variance of $\nabla^2\Bar{f}$
  • Lemma D.3: Bounds on bias and variance of UL direction
  • ...and 5 more