Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Mixed precision iterative refinement for linear inverse problems

James G. Nagy, Lucas Onisk

TL;DR

This work addresses the challenge of solving linear discrete ill-posed problems in mixed precision by applying iterative refinement to the Tikhonov-regularized least-squares problem $\min_x \{\|Ax-b\|^2 + \alpha^2\|x\|^2\}$ ($L=I$). It develops a filter-based interpretation of IR iterates through preconditioned Landweber with a Tikhonov-type preconditioner, derives exact spectral filter-factor expressions, and extends the analysis to three-precision arithmetic. The authors demonstrate that mixed-precision IR yields working accuracies within a few decimal places of double precision and often surpasses an AIR baseline, with the results depending on preconditioner precision and regularization strength. The findings support the practical use of MP-IR for large-scale inverse problems on modern hardware, providing guidance on precision choices and preconditioner design to balance accuracy and variability.

Abstract

This study investigates the iterative refinement method applied to the solution of linear discrete inverse problems by considering its application to the Tikhonov problem in mixed precision. Previous works on mixed precision iterative refinement methods for the solution of symmetric positive definite linear systems and least-squares problems have shown regularization to be a key requirement when computing low precision factorizations. For problems that are naturally severely ill-posed, we formulate the iterates of iterative refinement in mixed precision as a filtered solution using the preconditioned Landweber method with a Tikhonov-type preconditioner. Through numerical examples simulating various mixed precision choices, we showcase the filtering properties of the method and the achievement of comparable working accuracy of discrete inverse problems (i.e., to within a few decimal places in relative error) compared to results computed in double precision as well as another approximate iterative refinement method which we use as a benchmark.

Mixed precision iterative refinement for linear inverse problems

TL;DR

This work addresses the challenge of solving linear discrete ill-posed problems in mixed precision by applying iterative refinement to the Tikhonov-regularized least-squares problem (). It develops a filter-based interpretation of IR iterates through preconditioned Landweber with a Tikhonov-type preconditioner, derives exact spectral filter-factor expressions, and extends the analysis to three-precision arithmetic. The authors demonstrate that mixed-precision IR yields working accuracies within a few decimal places of double precision and often surpasses an AIR baseline, with the results depending on preconditioner precision and regularization strength. The findings support the practical use of MP-IR for large-scale inverse problems on modern hardware, providing guidance on precision choices and preconditioner design to balance accuracy and variability.

Abstract

This study investigates the iterative refinement method applied to the solution of linear discrete inverse problems by considering its application to the Tikhonov problem in mixed precision. Previous works on mixed precision iterative refinement methods for the solution of symmetric positive definite linear systems and least-squares problems have shown regularization to be a key requirement when computing low precision factorizations. For problems that are naturally severely ill-posed, we formulate the iterates of iterative refinement in mixed precision as a filtered solution using the preconditioned Landweber method with a Tikhonov-type preconditioner. Through numerical examples simulating various mixed precision choices, we showcase the filtering properties of the method and the achievement of comparable working accuracy of discrete inverse problems (i.e., to within a few decimal places in relative error) compared to results computed in double precision as well as another approximate iterative refinement method which we use as a benchmark.
Paper Structure (14 sections, 7 theorems, 63 equations, 4 figures, 4 tables, 5 algorithms)

This paper contains 14 sections, 7 theorems, 63 equations, 4 figures, 4 tables, 5 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. Filtered Solution of Linear Discrete Ill-posed Problems
  4. Landweber Iteration
  5. Iterative Refinement on the Tikhonov Problem
  6. Landweber with a Tikhonov-type Preconditioner
  7. Filtering Analysis of Iterative Refinement
  8. Reformulated Iterative Refinement on the Tikhonov problem
  9. Mixed precision IR on the Tikhonov problem
  10. Numerical Results and Preliminaries
  11. Effective filter factors
  12. Low precision simulation in MATLAB
  13. Examples and Results
  14. Conclusion

Key Result

Proposition 3.1

(Filtered solution of preconditioned Landweber) \newlabelprop10 The $k^{th}$ iterative solution of preconLand with preconditioner $M = D_MV_M^T$ as defined above may be written as with the $j^{th}$ filter factor per $k^{th}$ iterative step given by under the assumption that $V_M^TV_A = I = V_A^TV_M$.

Figures (4)

  • Figure 1: Spectra example: filter factors of standard Tikhonov \ref{['TikFF']} compared against filter factors of iterative refinement computed using: (a) the effective filter factors \ref{['effectiveFF']}, (b) the filter factors computed by Theorem \ref{['thm1']}\ref{['IR_DP_FF']}, and (c) the filter factors computed by Theorem \ref{['thm2']}\ref{['IR_MP_FF']}. The filter factors are compared using $\alpha^2 = 1e\text{-}2$ after $1$ iteration with $1\%$ noise.
  • Figure 2: MP-IR filter factors (bottom row) determined by Theorem \ref{['thm2']} with $(\text{Pr}_1,\text{Pr}_2,\text{Pr}_3)=(3,2,1)$ for $1\%$ noise compared to their entry-wise computations of $|\Phi^{(k)} - \Omega^{(k)}|$ (top row) at iteration numbers (a) 1, (b) 5, and (c) 10.
  • Figure 3: Spectra example: graph of RRE vs. iteration number for the MP-IR results for various precision combinations for $3\%$ noise and $\alpha^2=1e\text{-}3$. Pane (a) focuses on precision combination $(3,\,3,\,3)$ capturing the large variation in the yellow region compared to the other precision combinations. Pane (b) focuses on the variability of precision combinations $(3,\,3,\,1)$ and $(3,\,3,\,2)$, denoted in the blue region which is smaller than the yellow region in pane (a).
  • Figure 4: Image deblurring example: (a) true Hubble image ($256 \times 256$ pixels), (b) PSF ($31 \times 31$ pixels), (c) blurred and $1\%$ noised image ($256 \times 256$ pixels).

Theorems & Definitions (11)

  • Proposition 3.1
  • Proof 1
  • Lemma 4.1
  • Proof 2
  • Theorem 4.2
  • Corollary 4.3
  • Lemma 4.4
  • Proof 3
  • Proposition 4.5
  • Proof 4
  • ...and 1 more