Randomized flexible Krylov methods for $\ell_p$ regularization

TL;DR

This work tackles large-scale ill-posed inverse problems with sparsity via \\ell_p regularization, primarily \\ell_1, by developing randomized flexible Krylov methods to accelerate iteratively reweighted norm schemes. It introduces sketch-based preconditioning and sketch-to-precondition strategies, and two families of randomized flexible solvers (with convergence guarantees) that reduce per-iteration cost while maintaining accuracy. The methods are analyzed for monotonic descent and implemented in sketch-and-solve and sketch-to-precondition variants, applicable to tall-skinny and near-square systems. Numerical experiments on subset selection, deblurring, and tomography show speedups and competitive accuracy, validating the practical impact of combining randomized linear algebra with flexible Krylov subspaces for sparse inverse problems.

Abstract

The computation of sparse solutions of large-scale linear discrete ill-posed problems remains a computationally demanding task. A powerful framework in this context is the use of iteratively reweighted schemes, which are based on constructing a sequence of quadratic tangent majorants of the $\ell_2$-$\ell_1$ regularization functional (with additional smoothing to ensure differentiability at the origin), and solving them successively. Recently, flexible Krylov-Tikhonov methods have been used to partially solve each problem in the sequence efficiently. However, in order to guarantee convergence, the complexity of the algorithm at each iteration increases with respect to more traditional methods. We propose a randomized flexible Krylov method to alleviate the increase of complexity, which leverages the adaptability of the flexible Krylov subspaces with the efficiency of `sketch-and-solve' methods. A possible caveat of the mentioned methods is their memory requirements. In this case, one needs to rely instead on inner-outer schemes. In these scenarios, we propose a `sketch-to-precondition' method to speed up the convergence of each of the subproblems in the sequence. The performance of these algorithms is shown through a variety of numerical examples.

Figures (10)

• Figure 1: Example 1. Top row: exact and reconstructed coefficients using different methods. Bottom row: errors with respect to the exact solution.
• Figure 2: Example 1. First column: Relative error norm histories for different solvers against the number of iterations (for inner-outer schemes, we display the total number of iterations, not only the outer ones). Second column: value of the function \ref{['eq:smoothed']} at each approximate solution for $p=1$ and $\tau=10^{-10}$. Note that on the top left we show the values at each inner-iteration, but for the sake of clarity, the other plots display only the error norm at each outer iteration–indexed by the cumulative count of inner iterations– with linear interpolation applied between successive points.
• Figure 3: Example 1. Relative error norm histories (left) and regularization parameters (right) for different solvers against the total number of iterations. The regularization parameters are chosen at each iteration using the discrepancy principle.
• Figure 4: Example 2. Left: Exact image. Right: Noisy, blurred image.
• Figure 5: Example 2. First column: Relative error norm histories for randomized flexible iteratively reweighted method. Second column: Value of the non-linear objective function \ref{['eq:smoothed']} with $p=1$ and $\tau=10^{-10}$ at the approximate solutions obtained using different methods. Third column: close-up of the bottom of the plots in the second column. The new methods are compared to other standard and flexible Krylov solvers (top row), as well as other standard solvers for $\ell_1$ regularization (bottom row).

Theorems & Definitions (2)

• Proposition 1
• Proposition 2