A Proximal-Gradient Method for Solving Regularized Optimization Problems with General Constraints
Frank E. Curtis, Xiaoyi Qu, Daniel P. Robinson
TL;DR
The paper addresses regularized optimization with general constraints by developing a proximal-gradient method that explicitly incorporates the regularizer $r$ in the trial steps and avoids penalty-based constraint handling. It ensures feasible subproblems and employs a merit-based acceptance mechanism, yielding worst-case iteration guarantees and strong identifications: limit points are first-order KKT points under LICQ, with $O(\varepsilon^{-2})$ complexity under a stronger constraint qualification, active-set identification under strict complementarity, and manifold identification under partial smoothness. Numerical experiments on CUTEst problems and sparse canonical correlation analysis demonstrate competitive performance relative to an augmented Lagrangian approach, Bazinga, highlighting the method’s structure-preserving and scalable nature for nonconvex constrained optimization with general constraints. Overall, the framework offers a penalty-free, feasible-subproblem-providing approach with rigorous convergence guarantees and practical effectiveness for applications in data science, finance, signal processing, and imaging.
Abstract
We propose, analyze, and test a proximal-gradient method for solving regularized optimization problems with general constraints. The method employs a decomposition strategy to compute trial steps and uses a merit function to determine step acceptance or rejection. Under various assumptions, we establish a worst-case iteration complexity result, prove that limit points are first-order KKT points, and show that manifold identification and active-set identification properties hold. Preliminary numerical experiments on a subset of the CUTEst test problems and sparse canonical correlation analysis problems demonstrate the promising performance of our approach.