Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Group Projected Subspace Pursuit for Block Sparse Signal Reconstruction: Convergence Analysis and Applications

Roy Y. He, Haixia Liu, Hao Liu

TL;DR

The paper analyzes the convergence of Group Projected Subspace Pursuit (GPSP) for block-sparse signal recovery under Block Restricted Isometry Property (BRIP) with a small Block Restricted Isometry Constant (BRIC). It derives a sufficient condition for exact recovery and an error bound under perturbations, demonstrating GPSP's robustness. It compares GPSP's feature-selection criteria to other greedy algorithms, showing that Subspace Projection Criterion (SPC) is the key to GPSP's strong performance, with Request Magnitude Criterion (RMC) further improving robustness to noise. Extensive numerical experiments across heterogeneous blocks, inexact data, and real applications (face recognition and PDE identification) validate GPSP's superior accuracy and stability relative to BOMP, BOMPR, BSP, and BCoSaMP.

Abstract

In this paper, we present a convergence analysis of the Group Projected Subspace Pursuit (GPSP) algorithm proposed by He et al. [HKL+23] (Group Projected subspace pursuit for IDENTification of variable coefficient differential equations (GP-IDENT), Journal of Computational Physics, 494, 112526) and extend its application to general tasks of block sparse signal recovery. We prove that when the sampling matrix satisfies the Block Restricted Isometry Property (BRIP) with a sufficiently small Block Restricted Isometry Constant (BRIC), GPSP exactly recovers the true block sparse signals. When the observations are noisy, this convergence property of GPSP remains valid if the magnitude of true signal is sufficiently large. GPSP selects the features by subspace projection criterion (SPC) for candidate inclusion and response magnitude criterion (RMC) for candidate exclusion. We compare these criteria with counterparts of other state-of-the-art greedy algorithms. Our theoretical analysis and numerical ablation studies reveal that SPC is critical to the superior performances of GPSP, and that RMC can enhance the robustness of feature identification when observations contain noises. We test and compare GPSP with other methods in diverse settings, including heterogeneous random block matrices, inexact observations, face recognition, and PDE identification. We find that GPSP outperforms the other algorithms in most cases for various levels of block sparsity and block sizes, justifying its effectiveness for general applications.

Group Projected Subspace Pursuit for Block Sparse Signal Reconstruction: Convergence Analysis and Applications

TL;DR

The paper analyzes the convergence of Group Projected Subspace Pursuit (GPSP) for block-sparse signal recovery under Block Restricted Isometry Property (BRIP) with a small Block Restricted Isometry Constant (BRIC). It derives a sufficient condition for exact recovery and an error bound under perturbations, demonstrating GPSP's robustness. It compares GPSP's feature-selection criteria to other greedy algorithms, showing that Subspace Projection Criterion (SPC) is the key to GPSP's strong performance, with Request Magnitude Criterion (RMC) further improving robustness to noise. Extensive numerical experiments across heterogeneous blocks, inexact data, and real applications (face recognition and PDE identification) validate GPSP's superior accuracy and stability relative to BOMP, BOMPR, BSP, and BCoSaMP.

Abstract

In this paper, we present a convergence analysis of the Group Projected Subspace Pursuit (GPSP) algorithm proposed by He et al. [HKL+23] (Group Projected subspace pursuit for IDENTification of variable coefficient differential equations (GP-IDENT), Journal of Computational Physics, 494, 112526) and extend its application to general tasks of block sparse signal recovery. We prove that when the sampling matrix satisfies the Block Restricted Isometry Property (BRIP) with a sufficiently small Block Restricted Isometry Constant (BRIC), GPSP exactly recovers the true block sparse signals. When the observations are noisy, this convergence property of GPSP remains valid if the magnitude of true signal is sufficiently large. GPSP selects the features by subspace projection criterion (SPC) for candidate inclusion and response magnitude criterion (RMC) for candidate exclusion. We compare these criteria with counterparts of other state-of-the-art greedy algorithms. Our theoretical analysis and numerical ablation studies reveal that SPC is critical to the superior performances of GPSP, and that RMC can enhance the robustness of feature identification when observations contain noises. We test and compare GPSP with other methods in diverse settings, including heterogeneous random block matrices, inexact observations, face recognition, and PDE identification. We find that GPSP outperforms the other algorithms in most cases for various levels of block sparsity and block sizes, justifying its effectiveness for general applications.
Paper Structure (26 sections, 16 theorems, 101 equations, 9 figures, 3 tables, 1 algorithm)

This paper contains 26 sections, 16 theorems, 101 equations, 9 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminary
  3. Notations and definitions
  4. Review of block restricted isometry property
  5. GPSP and Main Results
  6. Convergence of GPSP
  7. Recovery distortion with perturbed measurements
  8. Outline of the proof for Theorem \ref{['thm_main']}
  9. Outline of the proof for Theorem \ref{['thm::perturb']}
  10. Relations to Other Greedy Algorithms
  11. Feature inclusion criteria: IPC vs. SPC
  12. Feature exclusion criteria: RCC vs. RMC
  13. Numerical Experiments
  14. Heterogeneous Gaussian blocks
  15. Reconstruction with inexact data
  16. ...and 11 more sections

Key Result

Lemma 1

Let $\mathbf{A}\in\mathbb{R}^{N\times GM}$ be a block matrix, and $\mathcal{T}_k$ be the collection of index sets with $k$ elements. For any positive integer $k\leq G$, define where $\sigma_{\max}(\cdot)$ and $\sigma_{\min}(\cdot)$ denote the maximal and minimal singular values, respectively. Then

Figures (9)

  • Figure 1: Two ways of measuring correlations between a vector $\mathbf{y}$ and subspaces spanned by $\mathbf{F}_1$ and $\mathbf{F}_2$. (a) Accumulated inner product. This criterion is used in BOMP li2018new, BCoSaMP zhang2019recovery, and BSP kamali2013block. (b) Projection to subspaces. This criterion is used in BOMPR fu2014block and GPSP he2023group.
  • Figure 2: (a) Plot of $C_{M,k}$\ref{['eq::main1']} for different values of BRIC $\delta_{M,2k}$. A sufficient condition for GPSP to converge is that $C_{M,k}<1$. (b) Plot of $G_{M,k}$\ref{['eq::main_perturb']}, which is the scaling factor for controlling the recovery distortion by the perturbation. (c) Plot of $F_{M,k}$\ref{['eq::main2']} for different values of BRIC $\delta_{M,2k}$. When $F_{M,k}<1$, GPSP shrinks the residual if the perturbation is sufficiently small. See Theorem \ref{['thm.inaccurate2']}.
  • Figure 3: Influence of the configuration of columns in a block on the correlation measured by inner-product \ref{['eq_inner_score']}. The subspaces spanned by the columns of $\mathbf{F}_g$ (green arrows) in (a) and (b) are identical. Since the column vectors are closer to $\mathbf{F}_g\cap\mathbf{d}^\perp$, the score by IPC \ref{['eq_inner_score']} for (b) is lower than that for (a); the block $\mathbf{F}_g$ is considered inferior in (b) compared to (a). In contrast, the score by SPC \ref{['eq_project_score']} remains the same for (a) and (b).
  • Figure 4: Comparison study on the frequency of exact recovery of GPSP, BCoSaMP, BOMP, BOMPR and BSP with random sampling matrices consisting of heterogeneous Gaussian blocks. From left to right, each column corresponds to the block size $M=5,8,10$, respectively. The first row shows the results without column normalization and the second row shows the results when the columns are normalized. In all the tested settings, GPSP consistently achieves the highest levels of accuracy.
  • Figure 5: Comparison study on the frequency of exact recovery of GPSP, BCoSaMP, BOMP, BOMPR and BSP with random sampling matrices consisting of heterogeneous Gaussian blocks and inexact data. Columns are normalized. The observed responses are perturbed by additive Gaussian noises with mean $0.0$ and standard deviation $\sigma>0$. From left to right, each column corresponds to the block size $M=5,8,10$, respectively. The first row shows the results when $\sigma=0.2$, and the second rows shows the results when $\sigma=1.0$. Although compromised by the perturbations, in all the tested settings, GPSP consistently achieves the highest level of accuracy.
  • ...and 4 more figures

Theorems & Definitions (29)

  • Definition 2.1: $k$-sparse vector
  • Definition 2.2: Block $k$-sparse vector
  • Definition 2.3: Block Restricted Isometry Property (BRIP)
  • Lemma 1: Equations (69) and (70) in eldar2009robust
  • Lemma 2: Lemma 1 in kamali2013block
  • Lemma 3: Lemma 2 in kamali2013block
  • Lemma 4: Lemma 3 in kamali2013block
  • Theorem 1
  • Theorem 2
  • Theorem 3
  • ...and 19 more