Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Regularized Operator Extrapolation Method For Stochastic Bilevel Variational Inequality Problems

Mohammad Khalafi, Digvijay Boob

TL;DR

This work addresses bilevel variational inequalities with stochastic monotone operators at both inner and outer levels by introducing Regularized Operator Extrapolation (R-OpEx), a single-loop method that blends Tikhonov regularization with operator extrapolation and requires only one operator evaluation per iteration. The authors establish explicit convergence guarantees across regimes, showing an $ ext{O}( abla ext{ε}^{-4})$-type rate for general nonsmooth BVIs and improvements to $ ext{O}( ext{ε}^{-2})$ with mini-batching when the inner VI is smooth, along with faster rates in the inner-smooth or fully deterministic inner settings; strong monotonicity at the outer level yields further improvements to $ ext{O}( ext{ε}^{-4/5})$ or $ ext{O}( ext{ε}^{-2/3})$ under suitable conditions. The method maintains a single iterate sequence without extra operator evaluations, and experimental results on VI-constrained optimization and traffic equilibrium problems confirm the theoretical guarantees and practical effectiveness. Overall, the paper advances the state of the art for stochastic BVIs by delivering best-known convergence guarantees and a simple, scalable algorithm that works for nonsmooth and smooth inner/outer operators alike. The work has potential impact on stochastic VI-constrained optimization, equilibrium problems, and meta-learning contexts where bilevel VI formulations arise.

Abstract

The bilevel variational inequality (BVI) problem is a general model that captures various optimization problems, including VI-constrained optimization and equilibrium problems with equilibrium constraints (EPECs). This paper introduces a first-order method for smooth or nonsmooth BVI with stochastic monotone operators at inner and outer levels. Our novel method, called Regularized Operator Extrapolation $(\texttt{R-OpEx})$, is a single-loop algorithm that combines Tikhonov's regularization with operator extrapolation. This method needs only one operator evaluation for each operator per iteration and tracks one sequence of iterates. We show that $\texttt{R-OpEx}$ gives $\mathcal{O}(ε^{-4})$ complexity in nonsmooth stochastic monotone BVI, where $ε$ is the error in the inner and outer levels. Using a mini-batching scheme, we improve the outer level complexity to $\mathcal{O}(ε^{-2})$ while maintaining the $\mathcal{O}(ε^{-4})$ complexity in the inner level when the inner level is smooth and stochastic. Moreover, if the inner level is smooth and deterministic, we show complexity of $\mathcal{O}(ε^{-2})$. Finally, in case the outer level is strongly monotone, we improve to $\mathcal{O}(ε^{-4/5})$ for general BVI and $\mathcal{O}(ε^{-2/3})$ when the inner level is smooth and deterministic. To our knowledge, this is the first work that investigates nonsmooth stochastic BVI with the best-known convergence guarantees. We verify our theoretical results with numerical experiments.

Regularized Operator Extrapolation Method For Stochastic Bilevel Variational Inequality Problems

TL;DR

This work addresses bilevel variational inequalities with stochastic monotone operators at both inner and outer levels by introducing Regularized Operator Extrapolation (R-OpEx), a single-loop method that blends Tikhonov regularization with operator extrapolation and requires only one operator evaluation per iteration. The authors establish explicit convergence guarantees across regimes, showing an -type rate for general nonsmooth BVIs and improvements to with mini-batching when the inner VI is smooth, along with faster rates in the inner-smooth or fully deterministic inner settings; strong monotonicity at the outer level yields further improvements to or under suitable conditions. The method maintains a single iterate sequence without extra operator evaluations, and experimental results on VI-constrained optimization and traffic equilibrium problems confirm the theoretical guarantees and practical effectiveness. Overall, the paper advances the state of the art for stochastic BVIs by delivering best-known convergence guarantees and a simple, scalable algorithm that works for nonsmooth and smooth inner/outer operators alike. The work has potential impact on stochastic VI-constrained optimization, equilibrium problems, and meta-learning contexts where bilevel VI formulations arise.

Abstract

The bilevel variational inequality (BVI) problem is a general model that captures various optimization problems, including VI-constrained optimization and equilibrium problems with equilibrium constraints (EPECs). This paper introduces a first-order method for smooth or nonsmooth BVI with stochastic monotone operators at inner and outer levels. Our novel method, called Regularized Operator Extrapolation , is a single-loop algorithm that combines Tikhonov's regularization with operator extrapolation. This method needs only one operator evaluation for each operator per iteration and tracks one sequence of iterates. We show that gives complexity in nonsmooth stochastic monotone BVI, where is the error in the inner and outer levels. Using a mini-batching scheme, we improve the outer level complexity to while maintaining the complexity in the inner level when the inner level is smooth and stochastic. Moreover, if the inner level is smooth and deterministic, we show complexity of . Finally, in case the outer level is strongly monotone, we improve to for general BVI and when the inner level is smooth and deterministic. To our knowledge, this is the first work that investigates nonsmooth stochastic BVI with the best-known convergence guarantees. We verify our theoretical results with numerical experiments.
Paper Structure (26 sections, 13 theorems, 109 equations, 5 figures, 1 table, 1 algorithm)

This paper contains 26 sections, 13 theorems, 109 equations, 5 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Outline:
  3. Notation and preliminaries
  4. Convergence criteria.
  5. Weak sharpness of operators.
  6. The Regularized operator extrapolation method
  7. Convergence analysis of R-OpEx method
  8. General nonsmooth and stochastic BVI
  9. Step-size policy for $\eta_k$ independent of $K$
  10. Smooth inner VI
  11. Numerical experiments
  12. VI-constrained optimization problem.
  13. Traffic equilibrium problem
  14. Conclusion and discussion
  15. Appendix
  16. ...and 11 more sections

Key Result

Lemma 1

Consider the problem eq:bilevel-vi with $\mu_H\geq 0$. Let $\{x_k\}_{k\geq 1}$ be a sequence generated by Algorithm alg:IRopex. Then we have the following relation for any $x\in X$ where $\Delta \mathfrak{O}_k = \mathfrak{F}_k - \mathfrak{F}_{k-1} + \eta_{k-1}[\mathfrak{H}_k-\mathfrak{H}_{k-1}]$ and $\delta_k^F = \mathfrak{F}_k - F(x_k)$, $\delta_k^F = \mathfrak{H}_k - H(x_k)$.

Figures (5)

  • Figure 1: Optimality and feasibility gaps for $10$ replication of Algorithm \ref{['alg:IRopex']} with $5\times 10^6$ iterations (monotone case)
  • Figure 2: Optimality and feasibility gaps for $10$ replication of Algorithm \ref{['alg:IRopex']} with $5\times 10^6$ iterations (strongly monotone case)
  • Figure 3: Traffic Network with 5 nodes and 7 links
  • Figure 4: Optimality and feasibility gaps for $10$ replication of Algorithm \ref{['alg:IRopex']} with $5\times 10^6$ iterations (monotone case)
  • Figure 5: Optimality and feasibility gaps for $10$ replication of Algorithm \ref{['alg:IRopex']} with $5\times 10^6$ iterations (strongly monotone case)

Theorems & Definitions (32)

  • Definition 1
  • Definition 2
  • Definition 3
  • Lemma 1
  • Theorem 1
  • Remark 1
  • Theorem 2
  • Remark 2
  • Remark 3
  • Remark 4
  • ...and 22 more