Nonmonotone Spectral Analysis for Variational Inclusions
Oday Hazaimah
TL;DR
The paper addresses solving variational inclusions $0\in T(x)$ where $T$ is set-valued in a real Hilbert space, a setting where classical gradient methods can stagnate on nonconvex problems. It proposes a spectral subgradient algorithm (SSG) that blends a spectral step size $\lambda_k=\frac{\|x_k-x_{k-1}\|^2}{\langle T(x_k)-T(x_{k-1}), x_k-x_{k-1}\rangle}$ with a nonmonotone line search, using the search direction $d_k=\pm T_k$ to accelerate convergence and allow temporary increases in the objective. The main contributions are the formulation of the SSG for variational inclusions, convergence guarantees (either finite termination or $\lim_{k\to\infty}\|T_k\|=0$ with accumulation points in $0\in T(\bar{x})$), and a condition for strong global convergence via a positive definite convex combination of generalized Jacobians. The results offer a robust algorithmic framework for large-scale, nonconvex, nonsmooth problems with potential impact across optimization, control, and machine learning contexts.
Abstract
Gradient descent algorithms perform well in convex optimization but can get tied for finding local minima in non-convex optimization. A robust method that combines a spectral approach with nonmonotone line search strategy for solving variational inclusion problems is proposed. Spectral properties using eigenvalues information are used for accelerating the convergence. Nonmonotonic behaviour is exhibited to relax descent property and escape local minima. Nonmonotone spectral conditions leverage adaptive search directions and global convergence for the proposed spectral subgradient algorithm.