On fundamental properties of high-order forward-backward envelope
Alireza Kabgani, Masoud Ahookhosh
TL;DR
This work analyzes the fundamental properties of the high-order forward-backward splitting mapping (HiFBS) and its envelope (HiFBE) for nonconvex composite optimization using high-order regularization with order $p>1$. It establishes boundedness of HiFBS on bounded domains and proves Hölder continuity for HiFBE, along with explicit Fréchet and Mordukhovich subdifferentials, enabling a precise differentiability characterization. By leveraging prox-regularity of $g$ and $p$-calmness, the authors show local single-valuedness and continuity of HiFBS, which in turn guarantees differentiability and weak smoothness of HiFBE near calm points, with explicit Hölder orders under mild smoothness assumptions. These analytical results connect HiFBE to high-order proximal theory and provide a rigorous foundation for gradient-based algorithms addressing nonconvex composite problems with weak smoothness.
Abstract
This paper studies the fundamental properties of the high-order forward-backward splitting mapping (HiFBS) and its associated forward-backward envelope (HiFBE) through the lens of high-order regularization for nonconvex composite functions. Specifically, we (i) establish the boundedness and uniform boundedness of HiFBS, along with the Hölder and Lipschitz continuity of HiFBE; (ii) derive an explicit form for the subdifferentials of HiFBE; and (iii) investigate necessary and sufficient conditions for the differentiability and weak smoothness of HiFBE under suitable assumptions. By leveraging the prox-regularity of $g$ and the concept of $p$-calmness, we further demonstrate the local single-valuedness and continuity of HiFBS, which in turn guarantee the differentiability of HiFBE in neighborhoods of calm points. This paves the way for the development of gradient-based algorithms tailored to nonconvex composite optimization problems.