Quantile-Based Randomized Kaczmarz for Corrupted Tensor Linear Systems

TL;DR

The paper tackles tensor regression under adversarial corruptions in the observation tensor by extending the Kaczmarz framework to tensors via the $t$-product. It introduces Quantile Tensor Randomized Kaczmarz (QTRK) and a Masked variant (mQTRK) to detect and avoid corrupted updates using residual quantiles, providing convergence guarantees for QTRK and discussing the practical behavior and limitations of masking. The authors offer theoretical analysis, extensive synthetic experiments, and a video deblurring application to demonstrate robustness to large, sparse corruptions and to illustrate the different tradeoffs between QTRK and mQTRK. The work advances scalable, corruption-robust tensor regression methods with potential impact on multi-modal imaging and video processing tasks.

Abstract

The reconstruction of tensor-valued signals from corrupted measurements, known as tensor regression, has become essential in many multi-modal applications such as hyperspectral image reconstruction and medical imaging. In this work, we address the tensor linear system problem $\mathcal{A} \mathcal{X}=\mathcal{B}$, where $\mathcal{A}$ is a measurement operator, $\mathcal{X}$ is the unknown tensor-valued signal, and $\mathcal{B}$ contains the measurements, possibly corrupted by arbitrary errors. Such corruption is common in large-scale tensor data, where transmission, sensory, or storage errors are rare per instance but likely over the entire dataset and may be arbitrarily large in magnitude. We extend the Kaczmarz method, a popular iterative algorithm for solving large linear systems, to develop a Quantile Tensor Randomized Kaczmarz (QTRK) method robust to large, sparse corruptions in the observations $\mathcal{B}$. This approach combines the tensor Kaczmarz framework with quantile-based statistics, allowing it to mitigate adversarial corruptions and improve convergence reliability. We also propose and discuss the Masked Quantile Randomized Kaczmarz (mQTRK) variant, which selectively applies partial updates to handle corruptions further. We present convergence guarantees, discuss the advantages and disadvantages of our approaches, and demonstrate the effectiveness of our methods through experiments, including an application for video deblurring.

Figures (6)

• Figure 1: Median relative residual errors of QTRK applied on a system ${\bm{\mathcal{A}}} {\bm{\mathcal{X}}} = {\bm{\mathcal{B}}}$ where ${\bm{\mathcal{A}}} \in {\mathbb{R}}^{25 \times 5 \times 10}$, ${\bm{\mathcal{B}}} \in {\mathbb{R}}^{25 \times 4 \times 10}$, and the corruptions are generated from $\mathcal{N}(100,20)$. In the left column plots, $\tilde{\beta} = 0.025$, the middle column plots $\tilde{\beta} = 0.075$, and the right column plots $\tilde{\beta} = 0.1$. In the top row plots, $\tilde{\beta}_{\text{row}} = 0.2$, the middle row plots, $\tilde{\beta}_{\text{row}} = 0.4$, and the bottom row plots $\tilde{\beta}_{\text{row}} = 0.8$.
• Figure 2: Median relative residual errors of QTRK applied on a system ${\bm{\mathcal{A}}} {\bm{\mathcal{X}}} = {\bm{\mathcal{B}}}$ where ${\bm{\mathcal{A}}} \in {\mathbb{R}}^{25 \times 5 \times 10}$, ${\bm{\mathcal{B}}} \in {\mathbb{R}}^{25 \times 4 \times 10}$, and the corruptions are generated from $\mathcal{N}(10,5)$. In the left column plots, $\tilde{\beta} = 0.025$, the middle column plots $\tilde{\beta} = 0.075$, and the right column plots $\tilde{\beta} = 0.1$. In the top row plots, $\tilde{\beta}_{\text{row}} = 0.2$, the middle row plots, $\tilde{\beta}_{\text{row}} = 0.4$, and the bottom row plots $\tilde{\beta}_{\text{row}} = 0.8$.
• Figure 3: Median relative residual errors of mQTRK applied on a system ${\bm{\mathcal{A}}} {\bm{\mathcal{X}}} = {\bm{\mathcal{B}}}$ where ${\bm{\mathcal{A}}} \in {\mathbb{R}}^{25 \times 5 \times 10}$, ${\bm{\mathcal{B}}} \in {\mathbb{R}}^{25 \times 4 \times 10}$, and the corruptions are generated from $\mathcal{N}(100,20)$. In the left column plots, $\tilde{\beta} = 0.025$, the middle column plots $\tilde{\beta} = 0.05$, and the right column plots $\tilde{\beta} = 0.075$. In the top row plots, $\tilde{\beta}_{\text{row}} = 0.2$, the second row plots, $\tilde{\beta}_{\text{row}} = 0.4$, the third row plots $\tilde{\beta}_{\text{row}} = 0.8$, and the bottom row plots $\tilde{\beta}_{\text{row}} = 1$.
• Figure 4: Median relative residual errors of mQTRK applied on a system ${\bm{\mathcal{A}}} {\bm{\mathcal{X}}} = {\bm{\mathcal{B}}}$ where ${\bm{\mathcal{A}}} \in {\mathbb{R}}^{25 \times 5 \times 10}$, ${\bm{\mathcal{B}}} \in {\mathbb{R}}^{25 \times 4 \times 10}$, and the corruptions are generated from $\mathcal{N}(10,5)$. In the left column plots, $\tilde{\beta} = 0.025$, the middle column plots $\tilde{\beta} = 0.05$, and the right column plots $\tilde{\beta} = 0.075$. In the top row plots, $\tilde{\beta}_{\text{row}} = 0.2$, the second row plots, $\tilde{\beta}_{\text{row}} = 0.4$, the third row plots $\tilde{\beta}_{\text{row}} = 0.8$, and the bottom row plots $\tilde{\beta}_{\text{row}} = 1$.
• Figure 5: Median relative residual errors of QTRK and mQTRK applied on the same system ${\bm{\mathcal{A}}} {\bm{\mathcal{X}}} = {\bm{\mathcal{B}}}$ where ${\bm{\mathcal{A}}} \in {\mathbb{R}}^{25 \times 5 \times 10}$ and ${\bm{\mathcal{B}}} \in {\mathbb{R}}^{25 \times 4 \times 10}$. In the top row plots, the corruptions are generated from $\mathcal{N}(100,20)$, and in the bottom row plots, the corruptions are generated from $\mathcal{N}(10,5)$. In the left column plots, $\tilde{\beta} = 0.025$, the middle column plots $\tilde{\beta} = 0.075$, and the right column plots $\tilde{\beta} = 0.1$. In all experiments, $q = 1 - \tilde{\beta}$.

Theorems & Definitions (21)

• Definition 1
• Lemma 1: castillo2024randomized
• Remark 1
• Definition 2
• Definition 3
• Definition 4
• Definition 5: Estimated Corruptions
• Lemma 2: Bounding the Quantile Value
• proof
• Lemma 3: Corrupted Row Selected