Convergence analysis of linearized $\ell_q$ penalty methods for nonconvex optimization with nonlinear equality constraints
Lahcen El Bourkhissi, Ion Necoara
TL;DR
The paper tackles nonconvex optimization with nonlinear equality constraints by formulating a linearized $\ell_q$ penalty method (qLP) that combines Gauss-Newton-like linearization of the objective and constraints with a dynamic quadratic regularization. It proves global asymptotic convergence to a penalty-critical point and derives a rate result showing an $\epsilon$-first-order solution is obtained in $O\left(\epsilon^{-2-(q-1)/q}\right)$ outer iterations, with a refined total complexity of $O\left(\epsilon^{-2-(q-1)/q-1/(3q-2)}\right)$ when using accelerated subproblem solves; the exponent is minimized near $q\approx1.33$, yielding about $O(\epsilon^{-2.74})$ complexity. The method yields a differentiable, strongly convex subproblem and employs an adaptive penalty parameter $\rho$, enabling robust performance without exact problem knowledge. Numerical experiments on CUTEst problems compare against Lipschitz penalty methods, illustrating improved feasibility and competitive efficiency, and highlighting the practical value of the $q$-parameter as a trade-off between exact and quadratic penalty regimes. Overall, qLP provides a versatile, theoretically-grounded tool for solving nonconvex problems with nonlinear constraints in contexts where non-differentiable subproblems are undesirable.
Abstract
In this paper, we consider nonconvex optimization problems with nonlinear equality constraints. We assume that the objective function and the functional constraints are locally smooth. To solve this problem, we introduce a linearized $\ell_q$ penalty based method, where $q \in (1,2]$ is the parameter defining the norm used in the construction of the penalty function. Our method involves linearizing the objective function and functional constraints in a Gauss-Newton fashion at the current iteration in the penalty formulation and introduces a quadratic regularization. This approach yields an easily solvable subproblem, whose solution becomes the next iterate. By using a novel dynamic rule for the choice of the regularization parameter, we establish that the iterates of our method converge to an $ε$-first-order solution in $\mathcal{O}(1/{ε^{2+ (q-1)/q}})$ outer iterations. Finally, we put theory into practice and evaluate the performance of the proposed algorithm by making numerical comparisons with existing methods from literature.