Learning Variational Inequalities from Data: Fast Generalization Rates under Strong Monotonicity
Eric Zhao, Tatjana Chavdarova, Michael Jordan
TL;DR
This work shows that fast generalization rates $O(1/ε)$ can be achieved when learning solutions to strongly monotone variational inequalities (VIs) by extending stability-based generalization arguments from convex optimization to VIs. It identifies two viable routes to stability: (i) leveraging small-domain coverings to stabilize the VI gap, and (ii) using potential-gap surrogates in integrable operator settings, including multi-player games. By addressing the intrinsic instability of the standard VI gap function, the authors derive $O(1/n)$ generalization bounds in several regimes, including fast rates for small domains and high-probability bounds for surrogate gaps under Bernstein conditions. The results unify and extend existing fast-rate analyses for convex minimization and strongly convex–concave games to the VI setting, with implications for learning equilibria and multi-agent systems where operators are learned from data.
Abstract
Variational inequalities (VIs) are a broad class of optimization problems encompassing machine learning problems ranging from standard convex minimization to more complex scenarios like min-max optimization and computing the equilibria of multi-player games. In convex optimization, strong convexity allows for fast statistical learning rates requiring only $Θ(1/ε)$ stochastic first-order oracle calls to find an $ε$-optimal solution, rather than the standard $Θ(1/ε^2)$ calls. This note provides a simple overview of how one can similarly obtain fast $Θ(1/ε)$ rates for learning VIs that satisfy strong monotonicity, a generalization of strong convexity. Specifically, we demonstrate that standard stability-based generalization arguments for convex minimization extend directly to VIs when the domain admits a small covering, or when the operator is integrable and suboptimality is measured by potential functions; such as when finding equilibria in multi-player games.