Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Power of $\ell_1$-Norm Regularized Kaczmarz Algorithms for High-Order Tensor Recovery

Katherine Henneberger, Jing Qin

TL;DR

This work develops regularized Kaczmarz algorithms for recovering high-order tensors from partial data by leveraging the power of the $\ell_1$-norm regularization. It introduces proximal operators for $\ell_1^p$ with closed-form solutions for $p=1,2,3,4$ and builds two main frameworks: sparse-tensor recovery (L1PK-S) and low-rank-tensor recovery (L1PK-L) using the $L$-based t-product and t-SVD. The paper provides convergence proofs for cyclic and randomized updates and extends the algorithms with block and accelerated variants to improve scalability. Through extensive synthetic and real-data experiments (including image destriping and 4D video deconvolution), the proposed methods demonstrate superior accuracy and robustness with practical guidance on parameter selection and computational complexity.

Abstract

Tensors serve as a crucial tool in the representation and analysis of complex, multi-dimensional data. As data volumes continue to expand, there is an increasing demand for developing optimization algorithms that can directly operate on tensors to deliver fast and effective computations. Many problems in real-world applications can be formulated as the task of recovering high-order tensors characterized by sparse and/or low-rank structures. In this work, we propose novel Kaczmarz algorithms with a power of the $\ell_1$-norm regularization for reconstructing high-order tensors by exploiting sparsity and/or low-rankness of tensor data. In addition, we develop both a block and an accelerated variant, along with a thorough convergence analysis of these algorithms. A variety of numerical experiments on both synthetic and real-world datasets demonstrate the effectiveness and significant potential of the proposed methods in image and video processing tasks, such as image sequence destriping and video deconvolution.

Power of $\ell_1$-Norm Regularized Kaczmarz Algorithms for High-Order Tensor Recovery

TL;DR

This work develops regularized Kaczmarz algorithms for recovering high-order tensors from partial data by leveraging the power of the -norm regularization. It introduces proximal operators for with closed-form solutions for and builds two main frameworks: sparse-tensor recovery (L1PK-S) and low-rank-tensor recovery (L1PK-L) using the -based t-product and t-SVD. The paper provides convergence proofs for cyclic and randomized updates and extends the algorithms with block and accelerated variants to improve scalability. Through extensive synthetic and real-data experiments (including image destriping and 4D video deconvolution), the proposed methods demonstrate superior accuracy and robustness with practical guidance on parameter selection and computational complexity.

Abstract

Tensors serve as a crucial tool in the representation and analysis of complex, multi-dimensional data. As data volumes continue to expand, there is an increasing demand for developing optimization algorithms that can directly operate on tensors to deliver fast and effective computations. Many problems in real-world applications can be formulated as the task of recovering high-order tensors characterized by sparse and/or low-rank structures. In this work, we propose novel Kaczmarz algorithms with a power of the -norm regularization for reconstructing high-order tensors by exploiting sparsity and/or low-rankness of tensor data. In addition, we develop both a block and an accelerated variant, along with a thorough convergence analysis of these algorithms. A variety of numerical experiments on both synthetic and real-world datasets demonstrate the effectiveness and significant potential of the proposed methods in image and video processing tasks, such as image sequence destriping and video deconvolution.
Paper Structure (22 sections, 12 theorems, 60 equations, 7 figures, 3 tables, 5 algorithms)

This paper contains 22 sections, 12 theorems, 60 equations, 7 figures, 3 tables, 5 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Proposed Algorithms
  4. Proximal Operator of L1p
  5. Sparse Tensor Recovery
  6. Low Rank Tensor Recovery
  7. Convergence Analysis
  8. Extensions
  9. Block Kaczmarz
  10. Accelerated Kaczmarz
  11. Numerical Experiments
  12. Synthetic Numerical Experiments
  13. Real data experiments
  14. Image sequence destriping
  15. 4D Video Deconvolution
  16. ...and 7 more sections

Key Result

Corollary 2.5

There exist several separability properties about the $L$-based t-product. Consider ${\mathcal{A}}\in\mathbb{R}^{n_1\times n_2 \times n_3\times ...\times n_m}$ and ${\mathcal{X}}\in\mathbb{R}^{n_2\times l\times n_3\times ...\times n_m}$.

Figures (7)

  • Figure 1: Visualization of the shrinkage of $\mathop{\mathrm{prox}}\nolimits_{\lambda||\cdot||^p_1}(x)$ for $x\in \mathbb{R}$ and $p=1,2,3,4$.
  • Figure 2: Convergence for different $p$ values. The sparse data is created with a sampling rate of $80\%$
  • Figure 3: Convergence under different linear transforms
  • Figure 4: Convergence with different block size $M$.
  • Figure 5: Accelerated versus regular convergence with parameters: $t = 1, \lambda = 0.001, p= 2$, block size $M= 25$.
  • ...and 2 more figures

Theorems & Definitions (30)

  • Definition 2.1
  • Definition 2.2
  • Definition 2.3
  • Definition 2.4
  • Corollary 2.5
  • Remark 2.6
  • Definition 2.7
  • Lemma 3.1
  • proof
  • Lemma 3.2
  • ...and 20 more