Separable Bregman Framework for Sparsity Constrained Nonlinear Optimization

TL;DR

Novel optimality conditions and algorithms are developed, which extend the previously proposed hard-thresholding algorithms and give a theoretical analysis of these algorithms and extend previous results on properties of iterative hard thresholding-like algorithms.

Abstract

This paper considers the minimization of a continuously differentiable function over a cardinality constraint. We focus on smooth and relatively smooth functions. These smoothness criteria result in new descent lemmas. Based on the new descent lemmas, novel optimality conditions and algorithms are developed, which extend the previously proposed hard-thresholding algorithms. We give a theoretical analysis of these algorithms and extend previous results on properties of iterative hard thresholding-like algorithms. In particular, we focus on the weighted $\ell_2$ norm, which requires efficient solution of convex subproblems. We apply our algorithms to compressed sensing problems to demonstrate the theoretical findings and the enhancements achieved through the proposed framework.

Figures (7)

• Figure 1: Comparison of different algorithmic frameworks
• Figure 2: Effect of choice of DSM in GPNP framework
• Figure 3: Computational cost of different algorithmic frameworks
• Figure 4: Computational costs of GPNP algorithms equipped with different DSMs
• Figure 5: Trajectory of $f(x_k)$

Theorems & Definitions (45)

• Definition 2.1
• Definition 2.2
• Definition 2.3: L-smooth adaptability bregman_descent
• Lemma 2.4: Descent Lemma first_order_amir_beck
• Lemma 2.5: Bregman or Extended Descent Lemma bregman_descent
• Theorem 3.1: L-Stationarity
• Proof 1
• Lemma 3.2: Bregman Stationarity
• Proof 2
• Definition 4.1