Block Matrix and Tensor Randomized Kaczmarz Methods for Linear Feasibility Problems

TL;DR

The paper tackles large-scale linear feasibility problems in both matrix and tensor settings by introducing a block randomized Kaczmarz method (B-MRK) and tensorized RK variants (TRK-L, TRK-LB) under the t-product. It provides linear-in-expectation convergence guarantees via generalized Hoffman constants (extending to matrices and tensors) and demonstrates that tensor-based TRK-L can offer tighter convergence bounds than its matrix counterpart under comparable conditions. The methods are validated through extensive experiments on Gaussian data and image deblurring tasks, illustrating block-size and step-size effects and practical gains over classic RK approaches. Overall, the work advances scalable, tensor-aware randomized projection methods for linear feasibility with broad applications in imaging and beyond.

Abstract

The randomized Kaczmarz methods are a popular and effective family of iterative methods for solving large-scale linear systems of equations, which have also been applied to linear feasibility problems. In this work, we propose a new block variant of the randomized Kaczmarz method, B-MRK, for solving linear feasibility problems defined by matrices. We show that B-MRK converges linearly in expectation to the feasible region.Furthermore, we extend the method to solve tensor linear feasibility problems defined under the tensor t-product. A tensor randomized Kaczmarz (TRK) method, TRK-L, is proposed for solving linear feasibility problems that involve mixed equality and inequality constraints. Additionally, we introduce another TRK method, TRK-LB, specifically tailored for cases where the feasible region is defined by linear equality constraints coupled with bound constraints on the variables. We show that both of the TRK methods converge linearly in expectation to the feasible region. Moreover, the effectiveness of our methods is demonstrated through numerical experiments on various Gaussian random data and applications in image deblurring.

Figures (9)

• Figure 1: Results of B-MRK for a Gaussian random system with $A\in\mathbb R^{(500+700)\times100}$ and $B\in\mathbb R^{(500+700)\times 7}$. Left: Iterations versus residual errors with block size $|\tau|=10$ and various step sizes. Right: Step size versus residual errors of B-MRK after 5000 iterations with $|\tau|=10$.
• Figure 2: Results of B-MRK for a Gaussian random system with $A\in\mathbb R^{(500+700)\times100}$ and $B\in\mathbb R^{(500+700)\times 7}$. Left: Iterations versus residual errors of B-MRK with various combinations of block sizes and step sizes. Right: Running time versus residual errors of B-MRK with various combinations of block sizes and step sizes.
• Figure 3: Results of B-MRK with various combinations of block sizes and step sizes for binary classification. Left: Running time versus residual errors for $A\in\mathbb R^{10000\times 100}$. Right: Running time versus residual errors for $A\in\mathbb R^{10000\times 500}$.
• Figure 4: Results of TRK-L for a Gaussian random system with $\mathcal{A}\in\mathbb R^{(50+70)\times 50\times 10}$ and $\mathcal{B}\in\mathbb R^{(50+70)\times 7\times 10}$. Left: Iterations versus residual errors with various step coefficients $\alpha$. Right: Step coefficient $\alpha$ versus residual errors after 5000 iterations.
• Figure 5: Comparison of TRK-L and B-MRK for a Gaussian random system with $\mathcal{A}\in\mathbb R^{(50+70)\times 50\times 10}$ and $\mathcal{B}\in\mathbb R^{(50+70)\times 7\times 10}$. Left: Iterations versus residual errors of TRK-L with step coefficient $\alpha=1.8$ and B-MRK with step size $t=2$. Right: Running time versus residual errors of TRK-L with step coefficients $\alpha=1.8$ and B-MRK with step size $t=2$.

Theorems & Definitions (16)

• Lemma 3.1
• proof
• Theorem 3.2
• proof
• Lemma 4.1
• proof
• Lemma 4.2
• proof
• Theorem 4.3
• proof