A Two-step Krasnosel'skii-Mann Algorithm with Adaptive Momentum and Its Applications to Image Denoising and Matrix Completion

TL;DR

The paper tackles convex optimization problems in imaging and data completion by casting them as fixed-point problems for averaged nonexpansive operators $T$. It introduces the Two-step Krasnosel'skii-Mann Algorithm (TKMA), a two-step fixed-point method that blends a momentum-enhanced KM update with a Picard step on $T^2$ via $T^{\theta_k}$. The authors prove convergence to a fixed point of $T$ and establish an $o(1/\sqrt{k})$ rate under a finite-variation condition on the momentum parameters $\{\theta_k\}$. Empirical results on image denoising and low-rank matrix completion show that TKMA outperforms FPPA, PGA, Fast KM, and Halpern variants, indicating strong practical utility for large-scale convex optimization in imaging.

Abstract

In this paper, we propose a Two-step Krasnosel'skii-Mann (KM) Algorithm (TKMA) with adaptive momentum for solving convex optimization problems arising in image processing. Such optimization problems can often be reformulated as fixed-point problems for certain operators, which are then solved using iterative methods based on the same operator, including the KM iteration, to ultimately obtain the solution to the original optimization problem. Prior to developing TKMA, we first introduce a KM iteration enhanced with adaptive momentum, derived from geometric properties of an averaged nonexpansive operator T, KM acceleration technique, and information from the composite operator T^2. The proposed TKMA is constructed as a convex combination of this adaptive-momentum KM iteration and the Picard iteration of T^2. We establish the convergence of the sequence generated by TKMA to a fixed point of T. Moreover, under specific assumptions on the adaptive momentum parameters, we prove that the algorithm achieves an o(1/k^{1/2}) convergence rate in terms of the distance between successive iterates. Numerical experiments demonstrate that TKMA outperforms the FPPA, PGA, Fast KM algorithm, and Halpern algorithm on tasks such as image denoising and low-rank matrix completion.

Figures (8)

• Figure 1: KM momentum acceleration for the fixed-point iteration of $\frac{1}{4}$-averaged nonexpansive operators.
• Figure 1: Comparison of the original image, noisy image, and denoised images obtained by the competing algorithms. (a) Original image of 'Cameraman'; (b) Noisy image with noise level $\sigma=15$; (c) Denoised image using TKMA; (d) Denoised image using FPPA; (e) Denoised image using Halpern algorithm with $\lambda_k=\frac{1}{k+1}$; (f) Denoised image using Halpern algorithm with $\lambda_k=\frac{1}{\varphi_k+1}$; (g) Denoised image using Fast KM algorithm.
• Figure 2: PSNR (left) and OFV (right) versus the iteration count of operator $T$ by TKMA, FPPA, Halpern algorithm with $\lambda_k=\frac{1}{k+1}$, Halpern algorithm with $\lambda_k=\frac{1}{\varphi_k+1}$ and Fast KM algorithm.
• Figure 3: Comparison of the original image, noisy image, and denoised images obtained by the competing algorithms. (a) Original image of 'Lighthouse'; (b) Noisy image with noise level $\sigma=25$; (c) Denoised image using TKMA; (d) Denoised image using FPPA; (e) Denoised image using Halpern algorithm with $\lambda_k=\frac{1}{k+1}$; (f) Denoised image using Halpern algorithm with $\lambda_k=\frac{1}{\varphi_k+1}$; (g) Denoised image using Fast KM algorithm.
• Figure 4: PSNR (left) and OFV (right) versus the iteration count of operator $T$ by TKMA, FPPA, Halpern algorithm with $\lambda_k=\frac{1}{k+1}$, Halpern algorithm with $\lambda_k=\frac{1}{\varphi_k+1}$ and Fast KM algorithm.

Theorems & Definitions (22)

• Lemma 2.1
• Lemma 3.1: Krasnosel'skiı̆-Mann theorem krasnosel1955twomann1953mean
• Lemma 3.2
• Theorem 3.3
• Proof 1
• Theorem 3.4
• Proposition 4.1
• Proof 2
• Lemma 4.2
• Proof 3