Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Solving, Tracking and Stopping Streaming Linear Inverse Problems

Nathaniel Pritchard, Vivak Patel

TL;DR

This work addresses solving large-scale linear inverse problems by reformulating them as streaming problems and applying memory-efficient Generalized Block Randomized Kaczmarz (GBRK) solvers. It develops a practical residual tracking framework with uncertainty sets under sub-Exponential residual models, proving moment convergence and estimator consistency, and introduces an adaptive moving-average scheme to manage tracking and stopping. The approach is validated on large-scale streaming collocation problems, achieving strong coverage of progress and substantial computational savings compared with naive periodic full-residual checks. The results enable reliable, scalable deployment of streaming solvers for industrial and scientific inverse problems, with provable stopping guarantees.

Abstract

In large-scale applications including medical imaging, collocation differential equation solvers, and estimation with differential privacy, the underlying linear inverse problem can be reformulated as a streaming problem. In theory, the streaming problem can be effectively solved using memory-efficient, exponentially-converging streaming solvers. In practice, a streaming solver's effectiveness is undermined if it is stopped before, or well-after, the desired accuracy is achieved. In special cases when the underlying linear inverse problem is finite-dimensional, streaming solvers can periodically evaluate the residual norm at a substantial computational cost. When the underlying system is infinite dimensional, streaming solver can only access noisy estimates of the residual. While such noisy estimates are computationally efficient, they are useful only when their accuracy is known. In this work, we rigorously develop a general family of computationally-practical residual estimators and their uncertainty sets for streaming solvers, and we demonstrate the accuracy of our methods on a number of large-scale linear problems. Thus, we further enable the practical use of streaming solvers for important classes of linear inverse problems.

Solving, Tracking and Stopping Streaming Linear Inverse Problems

TL;DR

This work addresses solving large-scale linear inverse problems by reformulating them as streaming problems and applying memory-efficient Generalized Block Randomized Kaczmarz (GBRK) solvers. It develops a practical residual tracking framework with uncertainty sets under sub-Exponential residual models, proving moment convergence and estimator consistency, and introduces an adaptive moving-average scheme to manage tracking and stopping. The approach is validated on large-scale streaming collocation problems, achieving strong coverage of progress and substantial computational savings compared with naive periodic full-residual checks. The results enable reliable, scalable deployment of streaming solvers for industrial and scientific inverse problems, with provable stopping guarantees.

Abstract

In large-scale applications including medical imaging, collocation differential equation solvers, and estimation with differential privacy, the underlying linear inverse problem can be reformulated as a streaming problem. In theory, the streaming problem can be effectively solved using memory-efficient, exponentially-converging streaming solvers. In practice, a streaming solver's effectiveness is undermined if it is stopped before, or well-after, the desired accuracy is achieved. In special cases when the underlying linear inverse problem is finite-dimensional, streaming solvers can periodically evaluate the residual norm at a substantial computational cost. When the underlying system is infinite dimensional, streaming solver can only access noisy estimates of the residual. While such noisy estimates are computationally efficient, they are useful only when their accuracy is known. In this work, we rigorously develop a general family of computationally-practical residual estimators and their uncertainty sets for streaming solvers, and we demonstrate the accuracy of our methods on a number of large-scale linear problems. Thus, we further enable the practical use of streaming solvers for important classes of linear inverse problems.
Paper Structure (26 sections, 20 theorems, 59 equations, 2 figures, 5 tables, 1 algorithm)

This paper contains 26 sections, 20 theorems, 59 equations, 2 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Notation
  3. Problem Formulation & Algorithm
  4. Generalized Block Randomized Kaczmarz (GBRK)
  5. Sub-Exponential Distribution
  6. Streaming solver with tracking and stopping
  7. Progress Tracking Point Estimate.
  8. Progress Tracking Uncertainty Set Estimate.
  9. Stopping Criterion
  10. Moving Average Logistics
  11. Consistency of Estimators and Uncertainty Sets
  12. Convergence of the Residuals' Moments
  13. Transformation of Variables
  14. Geometric Reduction in Error
  15. Control of the Random Rate and Random Iteration
  16. ...and 11 more sections

Key Result

Lemma 1

If given a random variable $Y$ with $\mathbb{E}[Y] = \mu$ such that $Y - \mu \sim \text{SE}(\sigma,\omega)$ then for constants $c_1,c_2$ where $c_2 \geq c_1 > 0$ it is the case that $c_1(Y -\mu) \sim \text{SE}(c_2\sigma,c_2 \omega)$.

Figures (2)

  • Figure 1: In the left plot, the black lines represent the upper and lower bounds of the uncertainty set, the green line represents $\rho_k^\lambda$, the red line represents $\hat{\rho}_k^\lambda$ for $\lambda = 300$ and the right plot represents the same values for $\lambda = 100$.
  • Figure :

Theorems & Definitions (38)

  • Remark 1
  • Remark 2
  • Definition 1
  • Lemma 1
  • Lemma 2
  • Lemma 3
  • proof
  • Theorem 1: patel2021implicit, Theorem 4.1
  • Lemma 4
  • proof
  • ...and 28 more