Efficient sampling for sparse Bayesian learning using hierarchical prior normalization

TL;DR

This work addresses the difficulty of posterior sampling in high-dimensional sparse Bayesian learning with hierarchical scale-mixture priors. By constructing hierarchical prior-normalizing transport maps (TMs) that push the joint SBL prior to a standard normal, the authors transform the posterior into a more tractable reference form and enable efficient MCMC in transformed coordinates, followed by back-mapping to the original space. They derive the maps analytically using a Knothe–Rosenblatt structure and a product form of the prior, yielding a block-diagonal transformation with explicit components for the hyperparameters and the conditional x|θ terms. Numerical experiments across toy problems, signal deconvolution, Burgers’ equation initial data recovery, and impulse-image reconstruction show substantial gains in sampling efficiency and exploration when using the prior-normalized posterior, often outperforming conventional Gibbs sampling in multimodal or highly correlated settings. The approach extends to broader scale-mixture priors and sets the stage for combining with normal-prior–tailored samplers and geometry-aware methods, offering a practical pathway for robust uncertainty quantification in sparse inverse problems.

Abstract

We introduce an approach for efficient Markov chain Monte Carlo (MCMC) sampling for challenging high-dimensional distributions in sparse Bayesian learning (SBL). The core innovation involves using hierarchical prior-normalizing transport maps (TMs), which are deterministic couplings that transform the sparsity-promoting SBL prior into a standard normal one. We analytically derive these prior-normalizing TMs by leveraging the product-like form of SBL priors and Knothe--Rosenblatt (KR) rearrangements. These transform the complex target posterior into a simpler reference distribution equipped with a standard normal prior that can be sampled more efficiently. Specifically, one can leverage the standard normal prior by using more efficient, structure-exploiting samplers. Our numerical experiments on various inverse problems -- including signal deblurring, inverting the non-linear inviscid Burgers equation, and recovering an impulse image -- demonstrate significant performance improvements for standard MCMC techniques.

Figures (17)

• Figure 1: Normalized densities of the univariate generalized gamma hyper-prior $\pi( \theta )$ and the conditionally Gaussian prior $\pi( x | \theta )$. The hyper-prior is illustrated for the parameters in \ref{['tab:parameters']} and the the conditional prior for $\theta$ equal to the mode (maximum value) of $\pi( \theta )$, which is $\max\{ 0, \vartheta ( [ r\beta - 1 ]/r )^{1/r} \}$.
• Figure 1: First components of the KR map $s$ and its inverse $t = s^{-1}$ for the parameters in \ref{['tab:parameters']}
• Figure 1: True signal and noisy, blurred observational data
• Figure 2: Contour plots of the posterior density $\pi^y( x, \theta )$ in \ref{['expl:toy_problem']} with $y=0.2$ and $\sigma^2 = 10^{-2.8}$. We use the parameter combinations $(r,\beta,\vartheta)$ as detailed in \ref{['tab:parameters']}.
• Figure 2: Contour plots of the prior-normalized posterior for the univariate toy problem in \ref{['expl:toy_problem']}. We use the same parameters as in \ref{['fig:contour_posterior_toy']} for the original posterior.

Theorems & Definitions (2)

• Example 2.1: Toy problem
• Remark 3.1