Iterative Implicit Gradients for Nonconvex Optimization with Variational Inequality Constraints
Harshal D. Kaushik, Ming Jin
TL;DR
This work targets constrained bilevel optimization where the inner problem is a variational inequality (VI) and the outer objective is nonconvex. It introduces an iterative implicit-gradient method based on the D-gap merit function and fixed-point formulations to compute the outer gradient without differentiating through the inner VI, enabling scalable optimization for problems with VI constraints. The authors provide rigorous error bounds between the estimated and true gradients and prove a non-asymptotic outer convergence rate of $\mathcal{O}(1/K)$ under standard regularity assumptions, with inner iterations converging at a linear rate. The approach broadens the toolbox for constrained, large-scale machine learning, offering practical guidance for meta-learning, hyperparameter optimization, and reinforcement learning scenarios involving VI constraints.
Abstract
We propose an optimization proxy in terms of iterative implicit gradient methods for solving constrained optimization problems with nonconvex loss functions. This framework can be applied to a broad range of machine learning settings, including meta-learning, hyperparameter optimization, large-scale complicated constrained optimization, and reinforcement learning. The proposed algorithm builds upon the iterative differentiation (ITD) approach. We extend existing convergence and rate analyses from the bilevel optimization literature to a constrained bilevel setting, motivated by learning under explicit constraints. Since solving bilevel problems using first-order methods requires evaluating the gradient of the inner-level optimal solution with respect to the outer variable (the implicit gradient), we develop an efficient computation strategy suitable for large-scale structures. Furthermore, we establish error bounds relative to the true gradients and provide non-asymptotic convergence rate guarantees.