Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the Resolution of Stochastic MPECs over Networks: Distributed Implicit Zeroth-Order Gradient Tracking Methods

Mohammadjavad Ebrahimi, Uday V. Shanbhag, Farzad Yousefian

TL;DR

This work addresses distributed stochastic MPECs over networks by developing zeroth-order gradient-tracking methods that rely on randomized smoothing to handle nonsmooth, nonconvex implicit objectives. The proposed schemes, $DiZS-GT^{\text{1s}}$ and $DiZS-GT^{\text{2s}}$, provide explicit nonasymptotic complexity guarantees for both single-stage and two-stage formulations, achieving the best-known centralized rates in the exact cases and offering improved dimension dependence in the inexact two-stage setting. The analysis leverages a central-difference like zeroth-order gradient approximation based on two evaluations of the lower-level VI solutions and a distributed gradient-tracking consensus mechanism to coordinate among agents. Numerical experiments on networks with varying connectivity corroborate the theory, showing robustness of the distributed methods and competitiveness with centralized approaches, especially as network connectivity increases. The results advance the design of scalable, provably efficient distributed algorithms for hierarchical stochastic optimization problems in networked systems.

Abstract

The mathematical program with equilibrium constraints (MPEC) is a powerful yet challenging class of constrained optimization problems, where the constraints are characterized by a parametrized variational inequality (VI) problem. While efficient algorithms for addressing MPECs and their stochastic variants (SMPECs) have been recently presented, distributed SMPECs over networks pose significant challenges. This work aims to develop fully iterative methods with complexity guarantees for resolving distributed SMPECs in two problem settings: (1) distributed single-stage SMPECs and (2) distributed two-stage SMPECs. In both cases, the global objective function is distributed among a network of agents that communicate cooperatively. Under the assumption that the parametrized VI is uniquely solvable, the resulting implicit problem in upper-level decisions is generally neither convex nor smooth. Under some standard assumptions, including the uniqueness of the solution to the VI problems and the Lipschitz continuity of the implicit global objective function, we propose single-stage and two-stage zeroth-order distributed gradient tracking optimization methods where the gradient of a smoothed implicit objective function is approximated using two (possibly inexact) evaluations of the lower-level VI solutions. In the exact setting of both the single-stage and two-stage problems, we achieve the best-known complexity bound for centralized nonsmooth nonconvex stochastic optimization. This complexity bound is also achieved (for the first time) for our method in addressing the inexact setting of the distributed two-stage SMPEC. In addressing the inexact setting of the single-stage problem, we derive an overall complexity bound, improving the dependence on the dimension compared to the existing results for the centralized SMPECs.

On the Resolution of Stochastic MPECs over Networks: Distributed Implicit Zeroth-Order Gradient Tracking Methods

TL;DR

This work addresses distributed stochastic MPECs over networks by developing zeroth-order gradient-tracking methods that rely on randomized smoothing to handle nonsmooth, nonconvex implicit objectives. The proposed schemes, and , provide explicit nonasymptotic complexity guarantees for both single-stage and two-stage formulations, achieving the best-known centralized rates in the exact cases and offering improved dimension dependence in the inexact two-stage setting. The analysis leverages a central-difference like zeroth-order gradient approximation based on two evaluations of the lower-level VI solutions and a distributed gradient-tracking consensus mechanism to coordinate among agents. Numerical experiments on networks with varying connectivity corroborate the theory, showing robustness of the distributed methods and competitiveness with centralized approaches, especially as network connectivity increases. The results advance the design of scalable, provably efficient distributed algorithms for hierarchical stochastic optimization problems in networked systems.

Abstract

The mathematical program with equilibrium constraints (MPEC) is a powerful yet challenging class of constrained optimization problems, where the constraints are characterized by a parametrized variational inequality (VI) problem. While efficient algorithms for addressing MPECs and their stochastic variants (SMPECs) have been recently presented, distributed SMPECs over networks pose significant challenges. This work aims to develop fully iterative methods with complexity guarantees for resolving distributed SMPECs in two problem settings: (1) distributed single-stage SMPECs and (2) distributed two-stage SMPECs. In both cases, the global objective function is distributed among a network of agents that communicate cooperatively. Under the assumption that the parametrized VI is uniquely solvable, the resulting implicit problem in upper-level decisions is generally neither convex nor smooth. Under some standard assumptions, including the uniqueness of the solution to the VI problems and the Lipschitz continuity of the implicit global objective function, we propose single-stage and two-stage zeroth-order distributed gradient tracking optimization methods where the gradient of a smoothed implicit objective function is approximated using two (possibly inexact) evaluations of the lower-level VI solutions. In the exact setting of both the single-stage and two-stage problems, we achieve the best-known complexity bound for centralized nonsmooth nonconvex stochastic optimization. This complexity bound is also achieved (for the first time) for our method in addressing the inexact setting of the distributed two-stage SMPEC. In addressing the inexact setting of the single-stage problem, we derive an overall complexity bound, improving the dependence on the dimension compared to the existing results for the centralized SMPECs.

Paper Structure

This paper contains 12 sections, 19 theorems, 127 equations, 2 figures, 3 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries and background
  3. Relevance to existing problem classes
  4. Challenges and their potential resolution
  5. Nonsmooth analysis and randomized smoothing
  6. Distributed single-stage stochastic MPECs
  7. Distributed two-stage stochastic MPECs
  8. Numerical experiments
  9. Distributed single-stage SMPECs
  10. Distributed two-stage SMPECs
  11. Concluding remarks
  12. Appendix

Key Result

Proposition 1

Let $x \in \mathbb{R}^n$ and let $f:\mathbb{R}^n \to \mathbb{R}$ be Lipschitz continuous on $\mathbb{R}^n$. Then, the following statements hold. (i) $\partial f(x)$ is a nonempty, convex, and compact set and $\|g\| \le L$ for any $g \in \partial f(x)$, where $L>0$ denotes the Lipschitz parameter of

Figures (2)

  • Figure 1: Comparison of Algorithm \ref{['alg:DZGT']} with ZSOL$_{\text{ncvx}}^{\text{1s}}$ in terms of the sample averaged global objective function and consensus error five different networks, each with 20 nodes.
  • Figure 2: Comparison of Algorithm \ref{['alg:DZGT-2stage']} with ZSOL$_{\text{ncvx}}^{\text{2s}}$ in terms of the sample averaged global objective function and consensus error five different networks, each with 20 nodes.

Theorems & Definitions (50)

  • Definition 1: Clarke generalized gradient clarke2008nonsmooth
  • Remark 1
  • Proposition 1: clarke2008nonsmooth
  • Definition 2: Approximate Clarke generalized gradient goldstein1977optimization
  • Lemma 1
  • Remark 2
  • Proposition 2
  • proof
  • Lemma 2: Lévy concentration on $\mathbb{S}$ wainwright2019high
  • Lemma 3: pu2021distributed
  • ...and 40 more