Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An efficient penalty decomposition algorithm for minimization over sparse symmetric sets

Ahmad Mousavi, Morteza Kimiaei, Saman Babaie-Kafaki, Vyacheslav Kungurtsev

TL;DR

This paper introduces PD-QN, an efficient penalty-decomposition algorithm for minimizing $f$ over the intersection of a sparse nonconvex set with a symmetric convex feasible region. The inner loop solves a closed-form $x$-update and a current-support $y$-update via a low-cost sparse projection, guided by an accelerated line search and four diagonal Hessian approximations. Global convergence is established under a gradient-growth condition weaker than Lipschitz continuity, with accumulation points being basic feasible and CC-M stationarity for the original problem; full-sequence convergence follows under a bounded-penalty regime and uniform strong convexity. Numerical experiments on 30 synthetic and data-driven problems (dimensions 10–500) show that PD-QN is competitive with state-of-the-art methods in efficiency, robustness, and strong stationarity, aided by warm-starts and stagnation recovery. The work advances both theory and practice by linking penalty-decomposition methods to CC-M optimality in the sparse symmetric setting and delivering practical, scalable algorithms for large-scale sparse optimization.

Abstract

This paper proposes an improved quasi-Newton penalty decomposition algorithm for the minimization of continuously differentiable functions, possibly nonconvex, over sparse symmetric sets. The method solves a sequence of penalty subproblems approximately via a two-block decomposition scheme: the first subproblem admits a closed-form solution without sparsity constraints, while the second subproblem is handled through an efficient sparse projection over the symmetric feasible set. Under a new assumption on the gradient of the objective function, weaker than global Lipschitz continuity from the origin, we establish that accumulation points of the outer iterates are basic feasible and cardinality-constrained Mordukhovich stationarity points. To ensure robustness and efficiency in finite-precision arithmetic, the algorithm incorporates several practical enhancements, including an enhanced line search strategy based on either backtracking or extrapolation, and four inexpensive diagonal Hessian approximations derived from differences of previous iterates and gradients or from eigenvalue-distribution information. Numerical experiments on a diverse benchmark of $30$ synthetic and data-driven test problems, including machine-learning datasets from the UCI repository and sparse symmetric instances with dimensions ranging from $10$ to $500$, demonstrate that the proposed algorithm is competitive with several state-of-the-art methods in terms of efficiency, robustness, and strong stationarity.

An efficient penalty decomposition algorithm for minimization over sparse symmetric sets

TL;DR

This paper introduces PD-QN, an efficient penalty-decomposition algorithm for minimizing over the intersection of a sparse nonconvex set with a symmetric convex feasible region. The inner loop solves a closed-form -update and a current-support -update via a low-cost sparse projection, guided by an accelerated line search and four diagonal Hessian approximations. Global convergence is established under a gradient-growth condition weaker than Lipschitz continuity, with accumulation points being basic feasible and CC-M stationarity for the original problem; full-sequence convergence follows under a bounded-penalty regime and uniform strong convexity. Numerical experiments on 30 synthetic and data-driven problems (dimensions 10–500) show that PD-QN is competitive with state-of-the-art methods in efficiency, robustness, and strong stationarity, aided by warm-starts and stagnation recovery. The work advances both theory and practice by linking penalty-decomposition methods to CC-M optimality in the sparse symmetric setting and delivering practical, scalable algorithms for large-scale sparse optimization.

Abstract

This paper proposes an improved quasi-Newton penalty decomposition algorithm for the minimization of continuously differentiable functions, possibly nonconvex, over sparse symmetric sets. The method solves a sequence of penalty subproblems approximately via a two-block decomposition scheme: the first subproblem admits a closed-form solution without sparsity constraints, while the second subproblem is handled through an efficient sparse projection over the symmetric feasible set. Under a new assumption on the gradient of the objective function, weaker than global Lipschitz continuity from the origin, we establish that accumulation points of the outer iterates are basic feasible and cardinality-constrained Mordukhovich stationarity points. To ensure robustness and efficiency in finite-precision arithmetic, the algorithm incorporates several practical enhancements, including an enhanced line search strategy based on either backtracking or extrapolation, and four inexpensive diagonal Hessian approximations derived from differences of previous iterates and gradients or from eigenvalue-distribution information. Numerical experiments on a diverse benchmark of synthetic and data-driven test problems, including machine-learning datasets from the UCI repository and sparse symmetric instances with dimensions ranging from to , demonstrate that the proposed algorithm is competitive with several state-of-the-art methods in terms of efficiency, robustness, and strong stationarity.
Paper Structure (49 sections, 13 theorems, 144 equations, 10 figures, 2 tables, 1 algorithm)

This paper contains 49 sections, 13 theorems, 144 equations, 10 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem Definition
  3. Related Work
  4. Main Contributions of our Work
  5. Algorithmic Features of our Methodology
  6. Strong Global Convergence Feature
  7. Our Computational Plans
  8. Paper Organization
  9. Supplemental Theory and Algorithmic Components
  10. Preliminaries and Methodological Foundations
  11. Notation and Foundational Concepts
  12. Symmetric Sets
  13. Optimality Conditions
  14. For Smooth Problems over Convex Sets
  15. For Problem \ref{['pr:p-original']}
  16. ...and 34 more sections

Key Result

lemma thmcounterlemma

Let $C \subseteq \mathbb{R}^n$ be a closed, convex, type-1 symmetric set, or a type-2 symmetric set. Then the Fréchet normal cone mapping $x \mapsto N_C^F(x)$ is outer semicontinuous on every face of $C$ determined by a fixed support pattern. In particular, $C$ satisfies CCP in the sense of kanzow2

Figures (10)

  • Figure 1: Performance profiles of PD-LM1-a and IHT in terms of nf2g (first and second columns) and sec (third and fourth columns), and with $q_{{{\mathop{\rm sol}}}}\le 10^{-6}$ (first and third columns) and $q_{{{\mathop{\rm sol}}}}\le 10^{-3}$ (second and fourth columns).
  • Figure 2: Performance profiles of PD-LM1-a and IHT in terms of nf2g (first and second columns) and sec (third and fourth columns), and with $\mathrm{rg}_{S}(x_{{{\mathop{\rm sol}}}})\le 10^{-6}$ (first and third columns) and $\mathrm{rg}_{S}(x_{{{\mathop{\rm sol}}}})\le 10^{-3}$ (second and fourth columns).
  • Figure 3: Performance profiles of PD-LM1-a and BFS in terms of nf2g (first and second columns) and sec (third and fourth columns), and with $q_{{{\mathop{\rm sol}}}}\le 10^{-6}$ (first and third columns) and $q_{{{\mathop{\rm sol}}}}\le 10^{-3}$ (second and fourth columns).
  • Figure 4: Performance profiles of PD-LM1-a and BFS in terms of nf2g (first and second columns) and sec (third and fourth columns), and with $\mathrm{rg}_{S}(x_{{{\mathop{\rm sol}}}})\le 10^{-6}$ (first and third columns) and $\mathrm{rg}_{S}(x_{{{\mathop{\rm sol}}}})\le 10^{-3}$ (second and fourth columns).
  • Figure 5: Performance profiles of PD-LM1-a and ZCWS in terms of nf2g (first and second columns) and sec (third and fourth columns), and with $q_{{{\mathop{\rm sol}}}}\le 10^{-6}$ (first and third columns) and $q_{{{\mathop{\rm sol}}}}\le 10^{-3}$ (second and fourth columns).
  • ...and 5 more figures

Theorems & Definitions (36)

  • remark thmcounterremark: Normal cones for the sparsity set
  • remark thmcounterremark: Normal cones for convex sets $C$
  • remark thmcounterremark: Cone-continuity for symmetric convex sets
  • lemma thmcounterlemma: CCP for Symmetric Convex Sets
  • theorem 1: Local minimizers are CC-AM (hence CC-M)
  • remark thmcounterremark: Support adjustment and restart mechanism
  • remark thmcounterremark: Finiteness of restart mechanisms
  • remark thmcounterremark: On the case $C = \infty$ and the Entrance Argument
  • remark thmcounterremark: Bounded penalty regime versus classical PD methods
  • lemma thmcounterlemma: Uniform bound under standard conditions
  • ...and 26 more