Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Projected Variable Smoothing for Weakly Convex Optimization and Supremum Functions

Sergio López-Rivera, Pedro Pérez-Aros, Emilio Vilches

TL;DR

This work develops a projected variable smoothing framework for minimizing the sum of a smooth function and the composition of a weakly convex function with a linear operator over a subspace. It introduces a projected gradient-like scheme with regularized, smoothed surrogates and proves a ${ t O}( rac{1}{ε^3})$ complexity to achieve $ε$-approximate optimality, with extensions to an epoch-based variant that accelerates practical convergence. A central theoretical contribution is the analysis of the Moreau envelope and proximity operator for supremum-of-weakly-convex functions, including closed-form solutions in key quadratic and affine cases. The framework is demonstrated across distributionally robust optimization, constrained LASSO, and max-dispersion problems, with numerical experiments illustrating trade-offs between epoch design and proximal-computation cost, and showing the method’s robustness beyond Lipschitz assumptions when iterates remain bounded.

Abstract

In this paper, we address two main topics. First, we study the problem of minimizing the sum of a smooth function and the composition of a weakly convex function with a linear operator on a closed vector subspace. For this problem, we propose a projected variable smoothing algorithm and establish a complexity bound of $\mathcal{O}(ε^{-3})$ to achieve an $ε$-approximate solution. Second, we investigate the Moreau envelope and the proximity operator of functions defined as the supremum of weakly convex functions, and we compute the proximity operator in two important cases. In addition, we apply the proposed algorithm for solving a distributionally robust optimization problem, the LASSO with linear constraints, and the max dispersion problem. We illustrate numerical results for the max dispersion problem.

A Projected Variable Smoothing for Weakly Convex Optimization and Supremum Functions

TL;DR

This work develops a projected variable smoothing framework for minimizing the sum of a smooth function and the composition of a weakly convex function with a linear operator over a subspace. It introduces a projected gradient-like scheme with regularized, smoothed surrogates and proves a complexity to achieve -approximate optimality, with extensions to an epoch-based variant that accelerates practical convergence. A central theoretical contribution is the analysis of the Moreau envelope and proximity operator for supremum-of-weakly-convex functions, including closed-form solutions in key quadratic and affine cases. The framework is demonstrated across distributionally robust optimization, constrained LASSO, and max-dispersion problems, with numerical experiments illustrating trade-offs between epoch design and proximal-computation cost, and showing the method’s robustness beyond Lipschitz assumptions when iterates remain bounded.

Abstract

In this paper, we address two main topics. First, we study the problem of minimizing the sum of a smooth function and the composition of a weakly convex function with a linear operator on a closed vector subspace. For this problem, we propose a projected variable smoothing algorithm and establish a complexity bound of to achieve an -approximate solution. Second, we investigate the Moreau envelope and the proximity operator of functions defined as the supremum of weakly convex functions, and we compute the proximity operator in two important cases. In addition, we apply the proposed algorithm for solving a distributionally robust optimization problem, the LASSO with linear constraints, and the max dispersion problem. We illustrate numerical results for the max dispersion problem.

Paper Structure

This paper contains 17 sections, 16 theorems, 109 equations, 1 figure, 4 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Algorithm Projected Variable Smoothing
  4. Variable smoothing with epochs
  5. Supremum of weakly convex functions
  6. Supremum functions of weakly convex with quadratic functions
  7. Supremum function weakly convex with affine functions
  8. Applications
  9. Distributionally Robust Optimization
  10. Constrained LASSO
  11. Max Dispersion Problems
  12. Numerical experiments
  13. Conclusions
  14. Appendix
  15. Penalty Method
  16. ...and 2 more sections

Key Result

theorem 1

In the context of problem prob_w_conv, assume that $g$ is $L_{g}$-Lipschitz. Then, the sequence defined by Algorithm alg_w_convex_sev satisfies where $\widetilde{C}:=\dfrac{\sqrt{2}\sqrt{L_{\nabla h}+C^{-1}\|A\|^{2}}}{\sqrt{2^{1-\alpha}-1}}\sqrt{F_{1}(x_{1})-F^{*}+C L_{g}^{2}}$.

Figures (1)

  • Figure 1: Performance of algorithms \ref{['alg_max_disp_1']} and \ref{['alg_max_disp_2']} for $\lambda=100$ (left) and $\lambda=200$ (right).

Theorems & Definitions (32)

  • theorem 1
  • proof
  • remark thmcounterremark
  • theorem 2
  • proof
  • theorem 3
  • proof
  • lemma thmcounterlemma
  • proof
  • theorem 4
  • ...and 22 more