Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Online multidimensional dictionary learning

Ferdaous Ait Addi, Abdeslem Hafid Bentbib, Khalide Jbilou

TL;DR

This work extends dictionary learning to multidimensional signals by adopting the tensor $t$-product, enabling online learning directly on tensors. It introduces two online strategies, a first-order PSGD and a second-order ISTA-based method, both tailored to tensor data, and uses tensor-OMP for sparse coding alongside a tensorized Cholesky factorization. To accelerate completion tasks, the authors adapt Anderson acceleration to ISTA (AISTA) and prove a tensor null-space property to support recovery guarantees. Empirically, the approach yields competitive completion performance on multidimensional data, often with faster training times (PSGD) and higher reconstruction accuracy (ISTA-based methods) than existing tensor methods like TNN and RTC. Overall, the framework demonstrates scalable, high-quality tensor completion and denoising with practical impact on large-scale multidimensional data analysis.

Abstract

Dictionary learning is a widely used technique in signal processing and machine learning that aims to represent data as a linear combination of a few elements from an overcomplete dictionary. In this work, we propose a generalization of the dictionary learning technique using the t-product framework, enabling efficient handling of multidimensional tensor data. We address the dictionary learning problem through online methods suitable for tensor structures. To effectively address the sparsity problem, we utilize an accelerated Iterative Shrinkage-Thresholding Algorithm (ISTA) enhanced with an extrapolation technique known as Anderson acceleration. This approach significantly improves signal reconstruction results. Extensive experiments prove that our proposed method outperforms existing acceleration techniques, particularly in applications such as data completion. These results suggest that our approach can be highly beneficial for large-scale tensor data analysis in various domains.

Online multidimensional dictionary learning

TL;DR

This work extends dictionary learning to multidimensional signals by adopting the tensor -product, enabling online learning directly on tensors. It introduces two online strategies, a first-order PSGD and a second-order ISTA-based method, both tailored to tensor data, and uses tensor-OMP for sparse coding alongside a tensorized Cholesky factorization. To accelerate completion tasks, the authors adapt Anderson acceleration to ISTA (AISTA) and prove a tensor null-space property to support recovery guarantees. Empirically, the approach yields competitive completion performance on multidimensional data, often with faster training times (PSGD) and higher reconstruction accuracy (ISTA-based methods) than existing tensor methods like TNN and RTC. Overall, the framework demonstrates scalable, high-quality tensor completion and denoising with practical impact on large-scale multidimensional data analysis.

Abstract

Dictionary learning is a widely used technique in signal processing and machine learning that aims to represent data as a linear combination of a few elements from an overcomplete dictionary. In this work, we propose a generalization of the dictionary learning technique using the t-product framework, enabling efficient handling of multidimensional tensor data. We address the dictionary learning problem through online methods suitable for tensor structures. To effectively address the sparsity problem, we utilize an accelerated Iterative Shrinkage-Thresholding Algorithm (ISTA) enhanced with an extrapolation technique known as Anderson acceleration. This approach significantly improves signal reconstruction results. Extensive experiments prove that our proposed method outperforms existing acceleration techniques, particularly in applications such as data completion. These results suggest that our approach can be highly beneficial for large-scale tensor data analysis in various domains.

Paper Structure

This paper contains 19 sections, 6 theorems, 88 equations, 4 figures, 6 algorithms.

Table of Contents

  1. Introduction
  2. Notations and definitions
  3. Notations
  4. Definitions
  5. Online Dictionary Learning
  6. Generalized dictionary learning
  7. Generalized Online Dictionary Learning
  8. First order method
  9. Sparse coding stage
  10. Dictionary update
  11. Second order method
  12. Multidimensional completion problem
  13. Iterative Shrinkage Thresholding with Anderson acceleration
  14. Sufficient Condition for Signal Recovery
  15. Numerical results
  16. ...and 4 more sections

Key Result

Lemma 1

Let $\mathcal{B} \in \mathbb{R}^{1 \times 1 \times I_{3} \times \dots \times I_{N}}$ be a tensor with positive entries. Then, the square root of $\mathcal{B}$ under the t-product exists and is given by $\sqrt{\mathcal{B}}$, whose frontal slices are defined as where $P = I_{3} \cdot I_{4} \cdot \dots \cdot I_{N}$.

Figures (4)

  • Figure 1: Illustrates the performance of the method on the completion problem applied to Tiffany, Lena, and Peppers images, with $20 \%$ and of the pixels available.
  • Figure 2: (a)-(b) -shows the error values as a function of iterations for the three methods applied to the completion problem of two different images with $30\%$ of pixels available.
  • Figure 3: ((a)-(b) -shows the error values as a function of iterations for the three methods applied to the completion problem of two different images with $20\%$ of pixels available.
  • Figure 4: (a)-(b) -shows the error values as a function of time for the online dictionary learning methods PGSD and ISTA.

Theorems & Definitions (23)

  • Definition 1
  • Definition 2
  • Definition 3
  • Definition 4
  • Definition 5
  • Definition 6
  • Definition 7
  • Definition 8
  • Definition 9
  • Lemma 1
  • ...and 13 more