Scalable iterative data-adaptive RKHS regularization

TL;DR

This paper introduces iDARR, a scalable iterative regularization framework that adaptively defines a data-driven RKHS (DA-RKHS) to solve large ill-posed linear inverse problems without strong prior norms. Central to the approach is a generalized Golub–Kahan bidiagonalization (gGKB) that builds RKHS-restricted Krylov subspaces and enables projected, norm-regularized solves whose stopping is guided by the L-curve (or discrepancy principle). The DA-RKHS norm penalizes components associated with small singular values and ensures the search lies in the function space of identifiability, yielding stable solutions across exponential and polynomial spectral decays. Numerical results on Fredholm integral equations and 2D image deblurring show that iDARR outperforms standard $l^2$ and $L^2$-based IR methods and is significantly faster than direct RKHS-Tikhonov approaches while maintaining accuracy and stability with decreasing noise.

Abstract

We present iDARR, a scalable iterative Data-Adaptive RKHS Regularization method, for solving ill-posed linear inverse problems. The method searches for solutions in subspaces where the true solution can be identified, with the data-adaptive RKHS penalizing the spaces of small singular values. At the core of the method is a new generalized Golub-Kahan bidiagonalization procedure that recursively constructs orthonormal bases for a sequence of RKHS-restricted Krylov subspaces. The method is scalable with a complexity of $O(kmn)$ for $m$-by-$n$ matrices with $k$ denoting the iteration numbers. Numerical tests on the Fredholm integral equation and 2D image deblurring show that it outperforms the widely used $L^2$ and $l^2$ norms, producing stable accurate solutions consistently converging when the noise level decays.

Figures (7)

• Figure 1: Singular values of $A$ and generalized eigenvalues of $(A^\top A, B)$ for kernels in \ref{['eq:kernels']}.
• Figure 2: Results in the case of exponentially decaying spectrum. Top-row: typical estimators of IR-l2, IR-L2, and iDARR when $nsr=0.5$ and their denoising of the output signal. The 2nd-top row: the residual $\|Ax_k-b\|_2$ as iteration number $k$ increases in one realization when $nsr=0.5$, as well as the boxplots of the stopping iteration numbers in the 100 simulations. The lower two rows: boxplots of the estimators' $L^2_\rho$ errors and loss function values in the 100 simulations.
• Figure 3: Results in the case of polynomial decaying spectrum. Top-row: typical estimators of IR-l2, IR-L2, and iDARR when $nsr=0.0625$ and their denoising of the output signal. The 2nd-top row: the residual $\|Ax_k-b\|_2$ as iteration number $k$ increases in one realization when $nsr=0.0625$, as well as the box plots of the stopping iteration numbers the 100 simulations. The lower two rows: box plots of the estimators' $L^2_\rho$ errors and loss function values in the 100 simulations.
• Figure 4: Computational time in 10 simulations with $m=500$.
• Figure 5: The true images, noisy images blurred by PRblurspeckle, and the corresponding PSFs.

Theorems & Definitions (11)

• Definition 2.1: Function space of identifiability
• Lemma 2.2
• Lemma 2.3: Characterization of the adaptive RKHS
• Theorem 2.4: Computation of RKHS norm
• Definition 4.1: RKHS-restricted Krylov subspace
• Theorem 4.2: Properties of gGKB
• Proposition 4.3
• Proposition 4.4: Orthogonality
• Proposition 4.5: gGKB termination number
• Proposition 4.6