Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Randomized Kaczmarz methods for t-product tensor linear systems with factorized operators

Alejandra Castillo, Jamie Haddock, Iryna Hartsock, Paulina Hoyos, Lara Kassab, Alona Kryshchenko, Kamila Larripa, Deanna Needell, Shambhavi Suryanarayanan, Karamatou Yacoubou-Djima

TL;DR

The paper addresses tensor regression with a factorized operator ${\bm{\mathcal{A}}}= {\bm{\mathcal{U}}}{\bm{\mathcal{V}}}$ under the $t$-product, and develops two randomized Kaczmarz variants, FacTBRK and FacTBREK, that exploit this structure to achieve linear convergence in expectation to the minimal-norm least-squares solution. The authors provide a rigorous convergence analysis through fundamental lemmas, establish convergence horizons for the TB-based methods, and prove a comprehensive factorized convergence theorem that covers both consistent and inconsistent outer systems. Numerical experiments on synthetic data and a twice-blurred MRI video demonstrate the practicality and efficiency of the tensor-based methods, and comparisons with matricized formulations highlight the benefits of preserving the tensor structure. The results extend prior RK theory to factorized tensor systems and have strong implications for scalable tensor regression and imaging applications where operator factors arise naturally. Overall, the work contributes structure-exploiting, provably convergent algorithms for solving large-scale tensor linear systems with broad relevance to imaging and data analysis.

Abstract

Randomized iterative algorithms, such as the randomized Kaczmarz method, have gained considerable popularity due to their efficacy in solving matrix-vector and matrix-matrix regression problems. Our present work leverages the insights gained from studying such algorithms to develop regression methods for tensors, which are the natural setting for many application problems, e.g., image deblurring. In particular, we extend the randomized Kaczmarz method to solve a tensor system of the form $\mathbf{\mathcal{A}}\mathcal{X} = \mathcal{B}$, where $\mathcal{X}$ can be factorized as $\mathcal{X} = \mathcal{U}\mathcal{V}$, and all products are calculated using the t-product. We develop variants of the randomized factorized Kaczmarz method for matrices that approximately solve tensor systems in both the consistent and inconsistent regimes. We provide theoretical guarantees of the exponential convergence rate of our algorithms, accompanied by illustrative numerical simulations. Furthermore, we situate our method within a broader context by linking our novel approaches to earlier randomized Kaczmarz methods.

Randomized Kaczmarz methods for t-product tensor linear systems with factorized operators

TL;DR

The paper addresses tensor regression with a factorized operator under the -product, and develops two randomized Kaczmarz variants, FacTBRK and FacTBREK, that exploit this structure to achieve linear convergence in expectation to the minimal-norm least-squares solution. The authors provide a rigorous convergence analysis through fundamental lemmas, establish convergence horizons for the TB-based methods, and prove a comprehensive factorized convergence theorem that covers both consistent and inconsistent outer systems. Numerical experiments on synthetic data and a twice-blurred MRI video demonstrate the practicality and efficiency of the tensor-based methods, and comparisons with matricized formulations highlight the benefits of preserving the tensor structure. The results extend prior RK theory to factorized tensor systems and have strong implications for scalable tensor regression and imaging applications where operator factors arise naturally. Overall, the work contributes structure-exploiting, provably convergent algorithms for solving large-scale tensor linear systems with broad relevance to imaging and data analysis.

Abstract

Randomized iterative algorithms, such as the randomized Kaczmarz method, have gained considerable popularity due to their efficacy in solving matrix-vector and matrix-matrix regression problems. Our present work leverages the insights gained from studying such algorithms to develop regression methods for tensors, which are the natural setting for many application problems, e.g., image deblurring. In particular, we extend the randomized Kaczmarz method to solve a tensor system of the form , where can be factorized as , and all products are calculated using the t-product. We develop variants of the randomized factorized Kaczmarz method for matrices that approximately solve tensor systems in both the consistent and inconsistent regimes. We provide theoretical guarantees of the exponential convergence rate of our algorithms, accompanied by illustrative numerical simulations. Furthermore, we situate our method within a broader context by linking our novel approaches to earlier randomized Kaczmarz methods.

Paper Structure

This paper contains 24 sections, 8 theorems, 62 equations, 17 figures, 1 table, 5 algorithms.

Table of Contents

  1. Introduction
  2. Notation and Definitions
  3. Related Work
  4. Extended Methods
  5. Block Methods
  6. Methods for Factorized Systems
  7. Tensor Methods
  8. Contributions
  9. Organization of the paper
  10. Convergence Analysis
  11. Fundamental Lemmas
  12. Tensor block randomized Kaczmarz (TBRK)
  13. Tensor block randomized extended Kaczmarz (TBREK)
  14. FacTBRK and FacTBREK
  15. Numerical Experiments
  16. ...and 9 more sections

Key Result

Theorem 1

Let ${\bm{\mathcal{U}}} \in {\mathbb{R}}^{m \times m_1 \times p}, {\bm{\mathcal{V}}} \in {\mathbb{R}}^{m_1 \times n \times p}$, and ${\bm{\mathcal{Y}}} \in {\mathbb{R}}^{m \times l \times p}.$ Let $\alpha_{{\bm{\mathcal{V}}}}, \alpha_{{\bm{\mathcal{U}}}}, \beta_{\bm{\mathcal{U}}}, \alpha_{\max}, \al

Figures (17)

  • Figure 1: Relative error $\|{\bm{\mathcal{X}}}^{(t)} - {\bm{\mathcal{X}}}^\ddagger\|_F/\|{\bm{\mathcal{X}}}^\ddagger\|_F$ vs iteration $t$ of FacTBRK and FacTBREK with outer block sizes $|\mu_t| \in \{1, 5, 10\}$ and inner system block size $|\nu_t| = 1$ on inconsistent tensor linear system (left) and consistent tensor linear system (right).
  • Figure 2: (Left) Relative error $\|{\bm{\mathcal{X}}}^{(t)} - {\bm{\mathcal{X}}}^\ddagger\|_F/\|{\bm{\mathcal{X}}}^\ddagger\|_F$ vs iteration $t$ of FacTBRK on consistent linear system and FacTBREK on inconsistent linear system with outer block sizes $|\mu_t| \in \{1, 5, 10\}$ and inner system block size $|\nu_t| = 1$. (Right) Relative error $\|{\bm{\mathcal{X}}}^{(t)} - {\bm{\mathcal{X}}}^\ddagger\|_F/\|{\bm{\mathcal{X}}}^\ddagger\|_F$ vs iteration $t$ of FacTBREK on consistent and inconsistent linear systems with outer block sizes $|\mu_t| \in \{1, 5, 10\}$ and inner system block size $|\nu_t| = 1$.
  • Figure 3: Relative error $\|{\bm{\mathcal{X}}}^{(t)} - {\bm{\mathcal{X}}}^\ddagger\|_F/\|{\bm{\mathcal{X}}}^\ddagger\|_F$ vs iteration $t$ of FacTBRK on consistent linear system when ${\bm{\mathcal{A}}}$ is under-determined, ${\bm{\mathcal{U}}}$ is over-determined and ${\bm{\mathcal{V}}}$ is under-determined. We consider outer block sizes $|\mu_t| \in \{1, 3, 5\}$ and inner system block size (Upper Left) $|\nu_t| = 1$, (Upper Right) $|\nu_t| = 3$, (Lower) $|\nu_t| = 5$.
  • Figure 4: Relative error $\|{\bm{\mathcal{X}}}^{(t)} - {\bm{\mathcal{X}}}^\ddagger\|_F/\|{\bm{\mathcal{X}}}^\ddagger\|_F$ vs iteration $t$ of FacTBRK on consistent linear system when ${\bm{\mathcal{A}}}$ is over-determined, ${\bm{\mathcal{U}}}$ is over-determined and ${\bm{\mathcal{V}}}$ is under-determined. We consider outer block sizes $|\mu_t| \in \{1, 5, 10, 15\}$ and inner system block size (Upper Left) $|\nu_t| = 1$, (Upper Right) $|\nu_t| = 5$, (Lower Left) $|\nu_t| = 10$, (Lower Right) $|\nu_t| = 15$.
  • Figure 5: Relative error $\|{\bm{\mathcal{X}}}^{(t)} - {\bm{\mathcal{X}}}^\ddagger\|_F/\|{\bm{\mathcal{X}}}^\ddagger\|_F$ vs iteration $t$ of FacTBRK on consistent linear system when ${\bm{\mathcal{A}}}$ is over-determined and both ${\bm{\mathcal{U}}}$ and ${\bm{\mathcal{V}}}$ are over-determined. We consider outer block sizes $|\mu_t| \in \{1, 5, 10, 20, 25\}$ and inner system block size (Upper Left) $|\nu_t| = 1$, (Upper Right) $|\nu_t| = 5$, (Lower Left) $|\nu_t| = 10$, (Lower Right) $|\nu_t| = 20$.
  • ...and 12 more figures

Theorems & Definitions (24)

  • Definition 1
  • Definition 2
  • Definition 3
  • Definition 4
  • Theorem 1
  • Remark 1
  • Remark 2
  • Remark 3
  • Lemma 1
  • proof
  • ...and 14 more