Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Leveraging joint sparsity in hierarchical Bayesian learning

Jan Glaubitz, Anne Gelb

TL;DR

This work addresses recovering multiple parameter vectors from jointly measured linear systems by leveraging joint sparsity within a hierarchical Bayesian framework. It introduces a joint-sparsity-promoting prior with shared hyper-parameters and develops MMV-IAS and MMV-GSBL algorithms that outperform traditional IAS/GSBL on MMV problems, including parallel MRI, while allowing uncertainty quantification for fixed hyper-parameters. The analysis covers computational complexity, convexity conditions, and convergence, showing that global convexity holds for certain hyper-parameter regimes and that joint sparsity becomes more influential as the number of measurement vectors grows. The approach is extensible to other hierarchical priors and non-linear models, with promising applications in medical imaging and remote sensing.

Abstract

We present a hierarchical Bayesian learning approach to infer jointly sparse parameter vectors from multiple measurement vectors. Our model uses separate conditionally Gaussian priors for each parameter vector and common gamma-distributed hyper-parameters to enforce joint sparsity. The resulting joint-sparsity-promoting priors are combined with existing Bayesian inference methods to generate a new family of algorithms. Our numerical experiments, which include a multi-coil magnetic resonance imaging application, demonstrate that our new approach consistently outperforms commonly used hierarchical Bayesian methods.

Leveraging joint sparsity in hierarchical Bayesian learning

TL;DR

This work addresses recovering multiple parameter vectors from jointly measured linear systems by leveraging joint sparsity within a hierarchical Bayesian framework. It introduces a joint-sparsity-promoting prior with shared hyper-parameters and develops MMV-IAS and MMV-GSBL algorithms that outperform traditional IAS/GSBL on MMV problems, including parallel MRI, while allowing uncertainty quantification for fixed hyper-parameters. The analysis covers computational complexity, convexity conditions, and convergence, showing that global convexity holds for certain hyper-parameter regimes and that joint sparsity becomes more influential as the number of measurement vectors grows. The approach is extensible to other hierarchical priors and non-linear models, with promising applications in medical imaging and remote sensing.

Abstract

We present a hierarchical Bayesian learning approach to infer jointly sparse parameter vectors from multiple measurement vectors. Our model uses separate conditionally Gaussian priors for each parameter vector and common gamma-distributed hyper-parameters to enforce joint sparsity. The resulting joint-sparsity-promoting priors are combined with existing Bayesian inference methods to generate a new family of algorithms. Our numerical experiments, which include a multi-coil magnetic resonance imaging application, demonstrate that our new approach consistently outperforms commonly used hierarchical Bayesian methods.
Paper Structure (27 sections, 3 theorems, 57 equations, 15 figures, 2 algorithms)

This paper contains 27 sections, 3 theorems, 57 equations, 15 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. The joint hierarchical Bayesian model
  3. The likelihood function
  4. The joint-sparsity promoting conditionally Gaussian prior
  5. The generalized gamma hyper-prior
  6. Bayesian inference
  7. The iterative alternating sequential algorithm
  8. Updating the parameter vectors
  9. Updating the hyper-parameters
  10. Proposed algorithm and its relationship to current methodology
  11. Uncertainty quantification
  12. Analysis: Complexity, convexity, and convergence
  13. Computational complexity
  14. Convexity of the objective function
  15. Extension to generalized sparse Bayesian learning
  16. ...and 12 more sections

Key Result

Theorem 4.1

\newlabelthm:convexity0 Let $\mathcal{G}$ be the objective function in eq:G and $\eta = r \beta - (L/2 + 1)$.

Figures (15)

  • Figure 1: First column: The first two of four piecewise constant signals with a common edge profile and noisy blurred measurements. Second column: Reconstructions of the signals using the existing IAS algorithm to separately recover them (blue triangles) and the proposed MMV-IAS algorithm to jointly recover them (green squares). See \ref{['sub:deblurring']} for more details.
  • Figure 1: Graphical representation of the hierarchical Bayesian model promoting joint sparsity for two ($L=2$) measurement and parameters vectors, $\mathbf{y}_1, \mathbf{y}_2$ and $\mathbf{x}_1, \mathbf{x}_2$, respectively. Shaded and plain circles represent observed and unobserved (hidden) random variables, respectively. The arrows indicate how the random variables influence each other: The parameter vectors $\mathbf{x}_1,\mathbf{x}_2$ are connected to the measurement vectors $\mathbf{y}_1,\mathbf{y}_2$, respectively, via the likelihood \ref{['eq:joint_likelihood']}; The common hyper-parameters $\boldsymbol{\theta}$ are connected to $\mathbf{x}_1, \mathbf{x}_2$ via the joint-sparsity-promoting prior \ref{['eq:joint_prior']}. Using common gamma hyper-parameters $\boldsymbol{\theta}$ (instead of separate ones for $\mathbf{x}_1, \mathbf{x}_2$) results in $R \mathbf{x}_1$ and $R \mathbf{x}_2$ having the same support.
  • Figure 1: Different reconstructions of the first (top row) and second (bottom row) of four piecewise constant signals with a common edge profile and noisy blurred measurements using the existing IAS/GSBL algorithm to separately recover them (blue triangles) and the proposed MMV-IAS/-GSBL algorithm to jointly recover them (green squares).
  • Figure 2: Normalized MAP estimate of the hyper-parameter $\theta$ for the first (top row) and second (bottom row) signal using the IAS and MMV-IAS algorithms with $r=\pm1$ and the GSBL and MMV-GSBL algorithms
  • Figure 3: The $99.9\%$ credible intervals (CIs) for the recovered first (top row) and second (bottom row) signal conditioned on the MAP estimate of the hyper-parameter vector $\boldsymbol{\theta}^{\rm MAP}$ using the IAS and MMV-IAS algorithm with $r=\pm1$ as well as the GSB and MMV-GSBL algorithm
  • ...and 10 more figures

Theorems & Definitions (15)

  • Remark 2.1: Complex-valued forward operators
  • Remark 2.2: Non-linear data models
  • Remark 2.3: Dependent measurement vectors
  • Remark 2.4
  • Remark 2.5: Other sparsity-promoting hierarchical priors
  • Remark 3.1: Extensions of the IAS algorithm
  • Remark 3.2: Full posterior sampling
  • Theorem 4.1: Convexity of the objective function
  • Remark 4.2: Convergence of MMV-IAS
  • Remark 5.1: Uncertainty quantification
  • ...and 5 more