Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Block Sparse Bayesian Learning: A Diversified Scheme

Yanhao Zhang, Zhihan Zhu, Yong Xia

TL;DR

A novel prior called Diversified Block Sparse Prior is introduced to characterize the widespread block sparsity phenomenon in real-world data and a diversified block sparse Bayesian learning method (DivSBL) is proposed, utilizing EM algorithm and dual ascent method for hyperparameter estimation.

Abstract

This paper introduces a novel prior called Diversified Block Sparse Prior to characterize the widespread block sparsity phenomenon in real-world data. By allowing diversification on intra-block variance and inter-block correlation matrices, we effectively address the sensitivity issue of existing block sparse learning methods to pre-defined block information, which enables adaptive block estimation while mitigating the risk of overfitting. Based on this, a diversified block sparse Bayesian learning method (DivSBL) is proposed, utilizing EM algorithm and dual ascent method for hyperparameter estimation. Moreover, we establish the global and local optimality theory of our model. Experiments validate the advantages of DivSBL over existing algorithms.

Block Sparse Bayesian Learning: A Diversified Scheme

TL;DR

A novel prior called Diversified Block Sparse Prior is introduced to characterize the widespread block sparsity phenomenon in real-world data and a diversified block sparse Bayesian learning method (DivSBL) is proposed, utilizing EM algorithm and dual ascent method for hyperparameter estimation.

Abstract

This paper introduces a novel prior called Diversified Block Sparse Prior to characterize the widespread block sparsity phenomenon in real-world data. By allowing diversification on intra-block variance and inter-block correlation matrices, we effectively address the sensitivity issue of existing block sparse learning methods to pre-defined block information, which enables adaptive block estimation while mitigating the risk of overfitting. Based on this, a diversified block sparse Bayesian learning method (DivSBL) is proposed, utilizing EM algorithm and dual ascent method for hyperparameter estimation. Moreover, we establish the global and local optimality theory of our model. Experiments validate the advantages of DivSBL over existing algorithms.
Paper Structure (48 sections, 5 theorems, 64 equations, 29 figures, 3 tables, 2 algorithms)

This paper contains 48 sections, 5 theorems, 64 equations, 29 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Block-based
  3. Pattern-based
  4. Diversified block sparse Bayesian model
  5. Diversified block sparse prior
  6. Diversified intra-block variance
  7. Diversified inter-block correlation
  8. Connections to classical models
  9. Connection to RVM
  10. Connection to BSBL
  11. Posterior estimation
  12. Bayesian inference: DivSBL algorithm
  13. EM formulation
  14. Diversified correlation matrices by dual ascent
  15. Explicit constraints with complete dual ascent
  16. ...and 33 more sections

Key Result

Proposition 3.1

Define an explicit weak constraint function $\zeta: \mathbb{R}^{n^2} \rightarrow \mathbb{R}$. For the constrained optimization problem: the stationary point $(\{\mathbf{B}_i^{k+1}\}_{i=1}^g, \{\lambda_i^k\}_{i=1}^g)$ of the Lagrange function under given multipliers $\{\lambda_i^k\}_{i=1}^g$ satisfies: Then there exists a constrained optimization problem with hidden weak constraint $\psi: \mathbb

Figures (29)

  • Figure 1: Directed acyclic graph of diversified block sparse hierarchical structure. Except for Measurements (blue nodes), which are known, all other nodes are parameters to estimate.
  • Figure 2: The gold dashed line shows the preset block, and the black shadow represents the actual position of the block with its true size.
  • Figure 3: The consistency of multiple experiments with homoscedastic signals for (a) NMSE (b) Correlation, and with heteroscedastic signals for (c) NMSE and (d) Correlation.
  • Figure 4: NMSE variation with changing preset block sizes.
  • Figure 5: $L=20$
  • ...and 24 more figures

Theorems & Definitions (10)

  • Proposition 3.1
  • Theorem 4.1
  • Lemma 4.2
  • Lemma 4.3
  • Theorem 4.4
  • proof
  • proof
  • proof
  • proof
  • proof