ItsDEAL: Inexact two-level smoothing descent algorithms for weakly convex optimization

TL;DR

The paper tackles weakly convex nonsmooth optimization by introducing a two-level smoothing framework that uses the high-order Moreau envelope (HOME) to create smooth approximations $\varphi_{\gamma}^{p}$ of a nonconvex objective $\varphi$. It develops ItsDEAL, an inexact descent scheme that combines an inexact lower-level proximal computation (HOPE) with an upper-level first-order method, and provides parameter-free variants such as HiGDA and IDEALS that exploit Hölderian smoothness and adaptive line searches. The authors prove differentiability and weak smoothness of HOME for $p\in(1,2]$, establish subsequential convergence under mild inexactness, and obtain global and linear convergence results under the KL property, along with a preliminary numerical study on robust sparse recovery showing promising performance. The framework offers a versatile toolkit for designing efficient first-order methods for broad classes of weakly convex problems with potential impact in sparse recovery and related applications, by enabling controlled smoothing and inexact proximal computations. Overall, the work advances theory and practice for inexact smoothing-based optimization beyond classical convex settings.

Abstract

This paper deals with nonconvex optimization problems via a two-level smoothing framework in which the high-order Moreau envelope (HOME) is applied to generate a smooth approximation of weakly convex cost functions. As such, the differentiability and weak smoothness of HOME are further studied, as is necessary for developing inexact first-order methods for finding its critical points. Building on the concept of the inexact two-level smoothing optimization (ItsOPT), the proposed scheme offers a versatile setting, called Inexact two-level smoothing DEscent ALgorithm (ItsDEAL), for developing inexact first-order methods: (i) solving the proximal subproblem approximately to provide an inexact first-order oracle of HOME at the lower-level; (ii) developing an upper inexact first-order method at the upper-level. In particular, parameter-free inexact descent methods (i.e., dynamic step-sizes and an inexact nonmonotone Armijo line search) are studied that effectively leverage the weak smooth property of HOME. Although the subsequential convergence of these methods is investigated under some mild inexactness assumptions, the global convergence and the linear rates are studied under the extra Kurdyka-Łojasiewicz (KL) property. In order to validate the theoretical foundation, preliminary numerical experiments for robust sparse recovery problems are provided which reveal a promising behavior of the proposed methods.