Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Variational Garrote for Sparse Inverse Problems

Kanghun Lee, Hyungjoon Soh, Junghyo Jo

Abstract

Sparse regularization plays a central role in solving inverse problems arising from incomplete or corrupted measurements. Different regularizers correspond to different prior assumptions about the structure of the unknown signal, and reconstruction performance depends on how well these priors match the intrinsic sparsity of the data. This work investigates the effect of sparsity priors in inverse problems by comparing conventional L1 regularization with the Variational Garrote (VG), a probabilistic method that approximates L0 sparsity through variational binary gating variables. A unified experimental framework is constructed across multiple reconstruction tasks including signal resampling, signal denoising, and sparse-view computed tomography. To enable consistent comparison across models with different parameterizations, regularization strength is swept across wide ranges and reconstruction behavior is analyzed through train-generalization error curves. Experiments reveal characteristic bias-variance tradeoff patterns across tasks and demonstrate that VG frequently achieves lower minimum generalization error and improved stability in strongly underdetermined regimes where accurate support recovery is critical. These results suggest that sparsity priors closer to spike-and-slab structure can provide advantages when the underlying coefficient distribution is strongly sparse. The study highlights the importance of prior-data alignment in sparse inverse problems and provides empirical insights into the behavior of variational L0-type methods across different information bottlenecks.

Variational Garrote for Sparse Inverse Problems

Abstract

Sparse regularization plays a central role in solving inverse problems arising from incomplete or corrupted measurements. Different regularizers correspond to different prior assumptions about the structure of the unknown signal, and reconstruction performance depends on how well these priors match the intrinsic sparsity of the data. This work investigates the effect of sparsity priors in inverse problems by comparing conventional L1 regularization with the Variational Garrote (VG), a probabilistic method that approximates L0 sparsity through variational binary gating variables. A unified experimental framework is constructed across multiple reconstruction tasks including signal resampling, signal denoising, and sparse-view computed tomography. To enable consistent comparison across models with different parameterizations, regularization strength is swept across wide ranges and reconstruction behavior is analyzed through train-generalization error curves. Experiments reveal characteristic bias-variance tradeoff patterns across tasks and demonstrate that VG frequently achieves lower minimum generalization error and improved stability in strongly underdetermined regimes where accurate support recovery is critical. These results suggest that sparsity priors closer to spike-and-slab structure can provide advantages when the underlying coefficient distribution is strongly sparse. The study highlights the importance of prior-data alignment in sparse inverse problems and provides empirical insights into the behavior of variational L0-type methods across different information bottlenecks.
Paper Structure (18 sections, 18 equations, 4 figures, 1 table)

This paper contains 18 sections, 18 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Theory
  3. Regularized Inverse Problems
  4. Sparse Linear Regression
  5. Regularizers as Prior Distributions
  6. Variational Garrote
  7. Connection to the Inverse Problems
  8. Experiments
  9. Training protocol.
  10. Comparing regularization strengths across models.
  11. Datasets
  12. Sound signals.
  13. Images.
  14. Signal Resampling
  15. Signal Denoising
  16. ...and 3 more sections

Figures (4)

  • Figure 1: Conceptual illustration of sparse reconstruction tasks. (a) Signal resampling: a synthetic sinusoidal signal (black) is observed through random subsampling (red markers) and reconstructed using sparsity-based regularization (red dashed line). The inset shows the magnitude of the discrete cosine transform (DCT) spectrum. (b) Signal denoising: a clean signal (black) corrupted by additive Gaussian noise (gray) and its reconstructed estimate (red dashed line). The inset shows the corresponding DCT spectra of the noisy and reconstructed signals.
  • Figure 2: Train–generalization ($\mathcal{E}_{\mathrm{train}}$ vs. $\mathcal{E}_{\mathrm{gen}}$) error curves for LASSO and Variational Garrote (VG). Top row: signal resampling experiments for the synthetic signal (a) and real flute signal (b). Bottom row: signal denoising experiments for the synthetic (c) and real (d) datasets. For each task, the left plot shows the relationship between training error and generalization error on log–log scales. Solid lines denote VG and dashed lines denote LASSO. Curve colors indicate different sampling ratios (top) or noise amplitudes (bottom). Markers indicate the hyperparameter setting achieving the minimum generalization error. The dotted diagonal line represents $\mathcal{E}_{\mathrm{train}}=\mathcal{E}_{\mathrm{gen}}$. The adjacent right plot explicitly shows this minimum generalization error as a function of the sampling ratio or noise amplitude.
  • Figure 3: Qualitative comparison of sparse-view CT reconstructions ($K=40$ projection angles). Rows correspond to four datasets: Shepp–Logan phantom, LIDC-IDRI chest CT, BraTS brain MRI, and Walnut CT. Columns show the ground truth image, reconstructions obtained with filtered back-projection (FBP), LASSO, and the Variational Garrote (VG), together with their corresponding absolute error maps ("Diff"). Brighter colors in the difference maps indicate larger reconstruction errors.
  • Figure 4: Reconstruction error versus angular resolution in sparse-view CT. Mean squared error (MSE) between reconstructed and ground-truth images is shown as a function of the number of projection angles $K$ for four datasets: (a) Shepp–Logan phantom, (b) LIDC-IDRI chest CT, (c) BraTS brain MRI, and (d) Walnut CT. Filtered back-projection (FBP), LASSO, and Variational Garrote (VG) are compared. Each point corresponds to the reconstruction obtained with the hyperparameter that minimizes MSE for the given $K$. Markers show the mean over 10 trials and error bars denote standard deviation.