Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Primal Methods for Variational Inequality Problems with Functional Constraints

Liang Zhang, Niao He, Michael Muehlebach

TL;DR

This paper tackles functional-constrained variational inequality problems where projection or linear-minimization oracles are costly. It introduces the Constrained Gradient Method (CGM), a primal, multiplier-free approach that updates via a quadratic program projected onto a local, sparse velocity polytope defined by active constraints. The authors prove convergence for both monotone and strongly monotone operators, showing CGM matches the operator-query complexity of projection-based methods while using cheaper QP-based oracles. Numerical experiments across 2D and high-dimensional matrix games illustrate CGM’s effectiveness in reducing both optimality gaps and constraint violations, starting from infeasible points. The work offers a practical, guidance-rich, projection-free alternative for functional-constrained VI problems with broad applications.

Abstract

Variational inequality problems are recognized for their broad applications across various fields including machine learning and operations research. First-order methods have emerged as the standard approach for solving these problems due to their simplicity and scalability. However, they typically rely on projection or linear minimization oracles to navigate the feasible set, which becomes computationally expensive in practical scenarios featuring multiple functional constraints. Existing efforts to tackle such functional constrained variational inequality problems have centered on primal-dual algorithms grounded in the Lagrangian function. These algorithms along with their theoretical analysis often require the existence and prior knowledge of the optimal Lagrange multipliers. In this work, we propose a simple primal method, termed Constrained Gradient Method (CGM), for addressing functional constrained variational inequality problems, without requiring any information on the optimal Lagrange multipliers. We establish a non-asymptotic convergence analysis of the algorithm for Minty variational inequality problems with monotone operators under smooth constraints. Remarkably, our algorithms match the complexity of projection-based methods in terms of operator queries for both monotone and strongly monotone settings, while using significantly cheaper oracles based on quadratic programming. Furthermore, we provide several numerical examples to evaluate the efficacy of our algorithms.

Primal Methods for Variational Inequality Problems with Functional Constraints

TL;DR

This paper tackles functional-constrained variational inequality problems where projection or linear-minimization oracles are costly. It introduces the Constrained Gradient Method (CGM), a primal, multiplier-free approach that updates via a quadratic program projected onto a local, sparse velocity polytope defined by active constraints. The authors prove convergence for both monotone and strongly monotone operators, showing CGM matches the operator-query complexity of projection-based methods while using cheaper QP-based oracles. Numerical experiments across 2D and high-dimensional matrix games illustrate CGM’s effectiveness in reducing both optimality gaps and constraint violations, starting from infeasible points. The work offers a practical, guidance-rich, projection-free alternative for functional-constrained VI problems with broad applications.

Abstract

Variational inequality problems are recognized for their broad applications across various fields including machine learning and operations research. First-order methods have emerged as the standard approach for solving these problems due to their simplicity and scalability. However, they typically rely on projection or linear minimization oracles to navigate the feasible set, which becomes computationally expensive in practical scenarios featuring multiple functional constraints. Existing efforts to tackle such functional constrained variational inequality problems have centered on primal-dual algorithms grounded in the Lagrangian function. These algorithms along with their theoretical analysis often require the existence and prior knowledge of the optimal Lagrange multipliers. In this work, we propose a simple primal method, termed Constrained Gradient Method (CGM), for addressing functional constrained variational inequality problems, without requiring any information on the optimal Lagrange multipliers. We establish a non-asymptotic convergence analysis of the algorithm for Minty variational inequality problems with monotone operators under smooth constraints. Remarkably, our algorithms match the complexity of projection-based methods in terms of operator queries for both monotone and strongly monotone settings, while using significantly cheaper oracles based on quadratic programming. Furthermore, we provide several numerical examples to evaluate the efficacy of our algorithms.
Paper Structure (20 sections, 4 theorems, 67 equations, 3 figures, 1 table, 4 algorithms)

This paper contains 20 sections, 4 theorems, 67 equations, 3 figures, 1 table, 4 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Variational Inequality Problems
  4. Constrained Gradient Descent
  5. The Constrained Gradient Method (CGM)
  6. Convergence Analysis
  7. The Monotone Setting
  8. The Strongly-Monotone Setting
  9. The Special Case with Convex Minimization
  10. Solving the Quadratic Program in CGM
  11. Single Constraint Function
  12. The Simplex Constraints
  13. The General Case
  14. Numerical Experiments
  15. 2D Examples with Ellipse Constraints
  16. ...and 5 more sections

Key Result

Lemma 1

Let $\gamma>1$. Under Assumptions asp:vip and asp:constraint, Algorithm algo:cgd-vip with $\epsilon\leq 4L_F^2$ and the choice of $\alpha>0$ such that $\alpha\eta_t\leq (\gamma - 1)/(\gamma + 1), \forall\, t=0,1,\cdots,T-1$ satisfies $\forall t=0,1,\cdots,T-1$,

Figures (3)

  • Figure 1: The trajectory of CGM on two 2D examples with varying $\alpha$. Each marker denotes the iterate at each iteration. The shaded area represents the feasible set and the black star denotes the optimal solution. The initialization point is (0.5,1) for both examples.
  • Figure 2: The optimality gap and feasibility of CGM on the matrix game with the quadratic constraint. Both are measured on the average iterates of the algorithm.
  • Figure 3: The optimality gap and feasibility of CGM (Algorithm \ref{['algo:cgm-sp']}) and projected gradient descent ascent (GDA) on the matrix game with the simplex constraint. Both are measured on the average iterates of the algorithm.

Theorems & Definitions (9)

  • Definition 1
  • Lemma 1
  • Theorem 1
  • proof
  • Theorem 2
  • proof
  • Theorem 3
  • proof
  • proof