An objective-function-free algorithm for general smooth constrained optimization
S. Bellavia, S. Gratton, B. Morini, Ph. L. Toint
TL;DR
The paper addresses constrained optimization when the objective value is not used, developing an objective-function-free OFFO method (ADIC) that switches between normal steps reducing infeasibility and tangential steps improving dual optimality. The tangential updates are AdaGrad-like, and the approach yields a global convergence rate of $\mathcal{O}(1/\sqrt{k+1})$ for the averaged criticality measures under a full-rank Jacobian assumption. It provides two concrete variants (LP-based and projection-based) and proves worst-case complexity bounds, complemented by numerical experiments on CUTEst problems showing strong noise robustness and favorable performance for the projection variant. The work offers a principled, robust framework for general smooth constrained optimization without objective evaluations, with practical implications for noisy-gradient settings and large-scale problems.
Abstract
A new algorithm for smooth constrained optimization is proposed that never computes the value of the problem's objective function and that handles both equality and inequality constraints. The algorithm uses an adaptive switching strategy between a normal step aiming at reducing constraint's infeasibility and a tangential step improving dual optimality, the latter being inspired by the AdaGrad-norm method. Its worst-case iteration complexity is analyzed, showing that the norm of the gradients generated converges to zero like O(1/\sqrt{k+1}) for problems with full-rank Jacobians. Numerical experiments show that the algorithm's performance is remarkably insensitive to noise in the objective function's gradient.