Adaptive Bregman-Kaczmarz: An Approach to Solve Linear Inverse Problems with Independent Noise Exactly

TL;DR

This work analyzes a block Bregman-Kaczmarz method for finite-dimensional linear inverse problems under an independent-noise model where each block query yields a fresh noisy right-hand side. It introduces an adaptive stepsize rule $\eta_k=\dfrac{\gamma\beta_k}{\gamma\beta_k+1}$ and proves that, despite fresh noise at every step, the iterates converge to the true solution, with an early linear phase and a later $\mathcal{O}(k^{-1})$-type decay in the Bregman distance; the convergence relies on a recursion in $\beta_k$ and the noise level $\sigma$, quantified via $\|\mathbf{A}\|_{\square}$. Since $\hat{x}$ is unknown, the paper provides data-driven heuristics to estimate the hyperparameters $\gamma$ and $\beta_0$ from a short pilot BK run with $\eta_k=1$, enabling practical deployment. Numerical experiments on synthetic data and a computerized tomography problem demonstrate that adaptive BK variants (aRSK/haRSK) outperform non-adaptive baselines, achieving reconstructions below the noise floor and improved SSIM/PSNR in CT, with convergence consistent with the theoretical rates.

Abstract

We consider the block Bregman-Kaczmarz method for finite dimensional linear inverse problems. The block Bregman-Kaczmarz method uses blocks of the linear system and performs iterative steps with these blocks only. We assume a noise model that we call independent noise, i.e. each time the method performs a step for some block, one obtains a noisy sample of the respective part of the right-hand side which is contaminated with new noise that is independent of all previous steps of the method. One can view these noise models as making a fresh noisy measurement of the respective block each time it is used. In this framework, we are able to show that a well-chosen adaptive stepsize of the block Bergman-Kaczmarz method is able to converge to the exact solution of the linear inverse problem. The plain form of this adaptive stepsize relies on unknown quantities (like the Bregman distance to the solution), but we show a way how these quantities can be estimated purely from given data. We illustrate the finding in numerical experiments and confirm that these heuristic estimates lead to effective stepsizes.

Figures (6)

• Figure 1: Comparison of the original decay and the estimate from Remark \ref{['rem:estimate-decay']}.
• Figure 2: A comparison of the five methods as described in Section \ref{['sec:synthetic-experiments']} for $m = 2000, n = 100,$ sparsity $s=10$, $\gamma_{aRK}=0.05, \gamma_{aRSK} = 0.1$, $\lambda=0.05$, $N_0=400, N_1=100$. From our heuristics we obtained $\Tilde{\gamma} = 0.0777$ and $\Tilde{\beta}_0 = 1022.60*10^{6}$. Left: Relative residual. Right: Relative error.
• Figure 3: Evaluation of the stepsize $\eta_{k}$ (left) and the parameter $\beta_{k}$ (right) in the runs of the adaptive methods in Figure \ref{['fig:example1']}.
• Figure 4: The effect of the block number $M$ on the relative error versus epochs for Algorithm \ref{['alg:BK']} (in the aRSK version) $m = 200, n = 100,$ sparsity $s=10$, $\gamma$ determined by grid search, $\beta_{0}$ exact. Left: Relative residual. Right: Relative error.
• Figure 5: Ground truth solution and reconstruction by the different methods.

Theorems & Definitions (15)

• definition 1
• definition 2
• definition 3
• definition 4
• theorem 5
• lemma 6
• proof
• lemma 7
• proof
• corollary 8