Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A double iteratively reweighted algorithm for solving group sparse nonconvex optimization models

Wanqin Nie, Kai Tu, Minglu Ye, Shuqin Sun

TL;DR

This work introduces a double iteratively reweighted algorithm (GIR) for solving nonconvex, nonsmooth group-sparsity optimization where both objective and constraint are built from concave compositions. By linearizing and convexifying the problem, it forms subproblems that are efficiently solved with ADMM, while ensuring feasibility and convergence to stationary points under mild assumptions, aided by a generalized Bregman distance for termination. The method extends to constrained group-sparse models and shows superior recovery and efficiency in numerical tests against related approaches. The approach has practical impact for robust, structured sparsity in high-dimensional data, with potential extensions to rate analysis and broader applications.

Abstract

In this paper, we propose a double iteratively reweighted algorithm to solve nonconvex and nonsmooth optimization problems, where both the objectives and constraint functions are formulated by concave compositions to promote group-sparse structures. At each iteration, we combine convex surrogate with first-order information to construct linearly constrained subproblems to handle the concavity of model. The corresponding subproblems are then solved by alternating direction method of multipliers to satisfy the specific stop criteria. In particular, under mild assumptions, we prove that our algorithm guarantees the feasibility of each subsequent iteration point, and the cluster point of the resulting feasible sequence is shown to be a stationary point. Additionally, we extend the group sparse optimization model, pioneer the application of the double iterative reweighted algorithm to solve constrained group sparse models (which exhibits superior efficiency), and incorporate a generalized Bregman distance to characterize the algorithm's termination conditions. Preliminary numerical experiments show the efficiency of the proposed method.

A double iteratively reweighted algorithm for solving group sparse nonconvex optimization models

TL;DR

This work introduces a double iteratively reweighted algorithm (GIR) for solving nonconvex, nonsmooth group-sparsity optimization where both objective and constraint are built from concave compositions. By linearizing and convexifying the problem, it forms subproblems that are efficiently solved with ADMM, while ensuring feasibility and convergence to stationary points under mild assumptions, aided by a generalized Bregman distance for termination. The method extends to constrained group-sparse models and shows superior recovery and efficiency in numerical tests against related approaches. The approach has practical impact for robust, structured sparsity in high-dimensional data, with potential extensions to rate analysis and broader applications.

Abstract

In this paper, we propose a double iteratively reweighted algorithm to solve nonconvex and nonsmooth optimization problems, where both the objectives and constraint functions are formulated by concave compositions to promote group-sparse structures. At each iteration, we combine convex surrogate with first-order information to construct linearly constrained subproblems to handle the concavity of model. The corresponding subproblems are then solved by alternating direction method of multipliers to satisfy the specific stop criteria. In particular, under mild assumptions, we prove that our algorithm guarantees the feasibility of each subsequent iteration point, and the cluster point of the resulting feasible sequence is shown to be a stationary point. Additionally, we extend the group sparse optimization model, pioneer the application of the double iterative reweighted algorithm to solve constrained group sparse models (which exhibits superior efficiency), and incorporate a generalized Bregman distance to characterize the algorithm's termination conditions. Preliminary numerical experiments show the efficiency of the proposed method.

Paper Structure

This paper contains 9 sections, 6 theorems, 85 equations, 5 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Notation and preliminaries
  3. Double iteratively reweighted algorithm for group sparsity models
  4. Subproblem computation
  5. Convergence analysis
  6. Numerical experiments
  7. Importance of new problem and algorithm
  8. Algorithm comparision
  9. Conclusion

Key Result

theorem 1

Suppose that Assumption assum hold, and that the sequences $\{x^{k,l}\}_{l\in\mathbb{N}}$ and $\{u^{k,l}\}_{l\in\mathbb{N}}$ are generated by Algorithm Algorithm2. Then for sufficiently large $l$, the pair $\left(x^{k,l},u^{k,l}\right)$ satisfies conditions zzzz1-zzzz3.

Theorems & Definitions (19)

  • definition thmcounterdefinition
  • definition thmcounterdefinition
  • remark thmcounterremark
  • remark thmcounterremark
  • remark thmcounterremark
  • theorem 1
  • proof
  • proposition thmcounterproposition
  • remark thmcounterremark
  • theorem 2
  • ...and 9 more