Single-loop variance reduction methods in Bregman setups for finite-sum structured variational inequalities
Wang Zhong-bao, Zhang Zhong-cheng
TL;DR
The paper tackles finite-sum variational inequalities by introducing a single-loop variance-reduced algorithm that leverages the Bregman distance and inertial updates. It establishes almost sure convergence under monotone operators, with an optimal $\mathcal{O}\left(\frac{\sqrt{M}}{\varepsilon}\right)$ complexity for achieving an $\varepsilon$-gap, and $\mathcal{O}\left(\frac{1}{\varepsilon^2}\right)$ in non-monotone settings, while also proving the first linear convergence rate in a Bregman VI framework under suitable assumptions. The proposed method uses only a single Bregman proximal update per iteration and employs batching to control variance, avoiding SPIDER-like double loops. Empirical results on matrix-game and non-monotone VI benchmarks corroborate the theoretical guarantees and show competitive or superior performance compared to existing variance-reduced VI methods.
Abstract
In this paper, we address variational inequalities (VI) with a finite sum structure by proposing a novel single-loop variance-reduced algorithm that incorporates the Bregman distance. Under the monotone setting, we establish the almost sure convergence of the proposed algorithm and prove that it achieves the optimal complexity of $\mathcal{O}\left(\frac{\sqrt{M}}{\varepsilon }\right)$ for finding an $\varepsilon$-gap. Furthermore, under the non-monotone setting, we derive a complexity of $\mathcal{O}\left(\frac{1}{\varepsilon^2 }\right)$ of the algorithm. Our proposed method yields complexity results that either match or improve the state-of-the-art complexity bounds reported in existing literature. Notably, this work is the first to rigorously establish the linear convergence rate of the algorithm for solving finite-sum variational inequalities in Bregman setups. Finally, we report two numerical experiments to validate the effectiveness and practical performance of our method.