A Nonlinear Acceleration Method for Iterative Algorithms

TL;DR

This paper tackles slow convergence and instability in iterative algorithms used for signal recovery. It introduces a nonlinear (NL) acceleration method and a Modified NL (MNL) variant to boost convergence speed and stabilize divergent iterations, underpinned by convergence analysis. The approach is demonstrated across a range of algorithms including IM, IMAT, IRLS, SL0, and ADMM for Lasso, with additional synergy from Chebyshev acceleration. Empirical results show consistent improvements in reconstruction quality and an expanded stability range, though some limitations arise for certain problem dimensions.

Abstract

Iterative methods have led to better understanding and solving problems such as missing sampling, deconvolution, inverse systems, impulsive and Salt and Pepper noise removal problems. However, the challenges such as the speed of convergence and or the accuracy of the answer still remain. In order to improve the existing iterative algorithms, a non-linear method is discussed in this paper. The mentioned method is analyzed from different aspects, including its convergence and its ability to accelerate recursive algorithms. We show that this method is capable of improving Iterative Method (IM) as a non-uniform sampling reconstruction algorithm and some iterative sparse recovery algorithms such as Iterative Reweighted Least Squares (IRLS), Iterative Method with Adaptive Thresholding (IMAT), Smoothed l0 (SL0) and Alternating Direction Method of Multipliers (ADMM) for solving LASSO problems family (including Lasso itself, Lasso-LSQR and group-Lasso). It is also capable of both accelerating and stabilizing the well-known Chebyshev Acceleration (CA) method. Furthermore, the proposed algorithm can extend the stability range by reducing the sensitivity of iterative algorithms to the changes of adaptation rate.

Figures (20)

• Figure 1: Block Diagram of IMATI algorithm with distortion operator $G$ and relaxation parameter of $\lambda$.
• Figure 2: SNR curves of the IM and the MNL, $OSR=8$, $LR=\frac{1}{3}$, $\lambda=2.2$.
• Figure 3: SNR curves of the IM and the MNL, $OSR=4$, $LR=\frac{1}{2}$, $\lambda=2$.
• Figure 4: SNR curves of the IM and the MNL, $OSR=8$, $LR=\frac{1}{3}$, $\lambda=2.2$.
• Figure 5: PSNR and IIQA curves of the IM and the MNL(+Clipping), $\lambda=2$, $OSR=4$, $LR=\frac{1}{3}$, image: Lenna.