Block Gauss-Seidel methods for t-product tensor regression

TL;DR

The authors extend randomized block Gauss-Seidel and Kaczmarz-type techniques to tensor regression under the t-product, proposing TRBGS, TRBAGS, and their factorized-operator counterparts FacTRBGS, FacTRBAGS. They establish linear convergence in expectation to the least-norm solution, develop pseudoinverse-free variants, and provide detailed proofs and numerical validations. Their methods accommodate both standard and factorized measurement operators, and they demonstrate practical efficacy on synthetic data and a video deblurring application. The work broadens tensor algorithms by leveraging column/row-slice updates and interlaced outer-inner schemes, with clear implications for large-scale multidimensional regression problems.

Abstract

Randomized iterative algorithms, such as the randomized Kaczmarz method and the randomized Gauss-Seidel method, have gained considerable popularity due to their efficacy in solving matrix-vector and matrix-matrix regression problems. Our present work leverages the insights gained from studying such algorithms to develop regression methods for tensors, which are the natural setting for many application problems, e.g., image deblurring. In particular, we extend two variants of the block-randomized Gauss-Seidel method to solve a t-product tensor regression problem. We additionally develop methods for the special case where the measurement tensor is given in factorized form. We provide theoretical guarantees of the exponential convergence rate of our algorithms, accompanied by illustrative numerical simulations.

Figures (9)

• Figure 1: Relative error $\|{\bm{\mathcal{X}}}^{(t)} - {\bm{\mathcal{X}}}^\ddagger\|_F/\|{\bm{\mathcal{X}}}^\ddagger\|_F$ and residual error $\|{\bm{\mathcal{A}}} {\bm{\mathcal{X}}}^{(t)} - {\bm{\mathcal{A}}}{\bm{\mathcal{X}}}^\ddagger\|_F$ versus iteration $t$ of TRBGS on a consistent linear system when ${\bm{\mathcal{A}}}$ is over-determined. We consider sampling block sizes $|\mu| \in \{1, 5, 10\}$ in each case.
• Figure 4: Relative error $\|{\bm{\mathcal{X}}}^{(t)} - {\bm{\mathcal{X}}}^\ddagger\|_F/\|{\bm{\mathcal{X}}}^\ddagger\|_F$ and residual error $\|{\bm{\mathcal{A}}} {\bm{\mathcal{X}}}^{(t)} - {\bm{\mathcal{A}}}{\bm{\mathcal{X}}}^\ddagger\|_F$ versus iteration $t$ of TRBAGS on a consistent linear system when ${\bm{\mathcal{A}}}$ is over-determined. We consider sampling block sizes $|\mu| \in \{1, 5, 10\}$ in each case.
• Figure 7: Relative error $\|{\bm{\mathcal{X}}}^{(t)} - {\bm{\mathcal{X}}}^\ddagger\|_F/\|{\bm{\mathcal{X}}}^\ddagger\|_F$ and residual error $\|{\bm{\mathcal{A}}} {\bm{\mathcal{X}}}^{(t)} - {\bm{\mathcal{A}}}{\bm{\mathcal{X}}}^\ddagger\|_F$ versus iteration $t$ for TRBGS and TRBAGS on a consistent linear system when ${\bm{\mathcal{A}}}$ is over-determined. The block size is $|\mu| = 5$.
• Figure 10: Relative error (rel err) and residual error (res err) versus iteration $t$ for the interlaced outer system ${\bm{\mathcal{U}}} {\bm{\mathcal{Z}}} = {\bm{\mathcal{B}}}$ and inner system ${\bm{\mathcal{V}}} {\bm{\mathcal{X}}} = {\bm{\mathcal{Z}}}$. Here, FacTRBGS and FacTRBAGS are applied to an inconsistent tensor system where ${\bm{\mathcal{A}}}$ is under-determined, ${\bm{\mathcal{U}}}$ is over-determined, and ${\bm{\mathcal{V}}}$ is under-determined. The block size is kept constant at $|\mu| = 5$.
• Figure 13: Deblurring of twice sequentially blurred video data using the TRBGS and TRBAGS. In (a) and (b), we have frames from the original video (top row), twice blurred frames (second row), frames recovered using TRBGS and TRBAGS (third and fourth row) respectively. The least norm solution is show on the bottom row. In (c), we show the residual error of TRBGS and TRBAGS iterates for two different initializations.

Theorems & Definitions (16)

• Definition 1
• Theorem 1
• Theorem 2
• Theorem 3
• Theorem 4
• Lemma 1
• proof
• Lemma 2
• proof
• Lemma 3