Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Higher order multi-dimension reduction methods via Einstein product

Alaeddine Zahir, Khalide Jbilou, Ahmed Ratnani

TL;DR

This work advances dimensionality reduction by extending graph-based matrix methods to multi-dimensional data through the Einstein product, preserving tensor structure and avoiding vectorization. It develops tensor-valued generalizations of PCA, LPP, ONPP, OLPP, NPP, Laplacian Eigenmaps, and LLE, along with kernelized and supervised variants, including multi-weight and repulsion enhancements. The framework yields corresponding eigen-tensor or left-singular-tensor solutions and provides out-of-sample extensions for practical use. Experimental results on MNIST and GTDB demonstrate competitive performance and highlight the benefits of maintaining multi-dimensional structure for color images and high-dimensional data.

Abstract

This paper explores the extension of dimension reduction (DR) techniques to the multi-dimension case by using the Einstein product. Our focus lies on graph-based methods, encompassing both linear and nonlinear approaches, within both supervised and unsupervised learning paradigms. Additionally, we investigate variants such as repulsion graphs and kernel methods for linear approaches. Furthermore, we present two generalizations for each method, based on single or multiple weights. We demonstrate the straightforward nature of these generalizations and provide theoretical insights. Numerical experiments are conducted, and results are compared with original methods, highlighting the efficiency of our proposed methods, particularly in handling high-dimensional data such as color images.

Higher order multi-dimension reduction methods via Einstein product

TL;DR

This work advances dimensionality reduction by extending graph-based matrix methods to multi-dimensional data through the Einstein product, preserving tensor structure and avoiding vectorization. It develops tensor-valued generalizations of PCA, LPP, ONPP, OLPP, NPP, Laplacian Eigenmaps, and LLE, along with kernelized and supervised variants, including multi-weight and repulsion enhancements. The framework yields corresponding eigen-tensor or left-singular-tensor solutions and provides out-of-sample extensions for practical use. Experimental results on MNIST and GTDB demonstrate competitive performance and highlight the benefits of maintaining multi-dimensional structure for color images and high-dimensional data.

Abstract

This paper explores the extension of dimension reduction (DR) techniques to the multi-dimension case by using the Einstein product. Our focus lies on graph-based methods, encompassing both linear and nonlinear approaches, within both supervised and unsupervised learning paradigms. Additionally, we investigate variants such as repulsion graphs and kernel methods for linear approaches. Furthermore, we present two generalizations for each method, based on single or multiple weights. We demonstrate the straightforward nature of these generalizations and provide theoretical insights. Numerical experiments are conducted, and results are compared with original methods, highlighting the efficiency of our proposed methods, particularly in handling high-dimensional data such as color images.
Paper Structure (28 sections, 12 theorems, 67 equations, 2 figures, 2 tables, 11 algorithms)

This paper contains 28 sections, 12 theorems, 67 equations, 2 figures, 2 tables, 11 algorithms.

Table of Contents

  1. Introduction
  2. Dimension reduction methods in matrix case
  3. The Einstein product and its properties
  4. Multidimensional reduction
  5. Generalization of ONPP
  6. Multi-weight ONPP
  7. Generalization of OLPP
  8. Multi-weight OLPP
  9. Generalization of LPP
  10. Generalization of NPP
  11. Generalization of Laplacian Eigenmap
  12. Projection on out of Sample Data
  13. Generalization of LLE
  14. Computing the weights
  15. Computing the projected data
  16. ...and 13 more sections

Key Result

Proposition 3.6

Let $\mathcal{X} \in \mathbb{R}^{I_1 \times \ldots \times I_M \times J_1 \times \ldots \times J_N}$ and $\mathcal{U} \in \mathbb{R}^{I_1 \times \ldots \times I_M \times I_1 \times \ldots \times I_M}$ be a unitary tensor. Then

Figures (2)

  • Figure 6.1: Example of images of one person in the GTDB dataset.
  • Figure 6.2: Performance of methods on different subspace dimension.

Theorems & Definitions (25)

  • Definition 3.1: m-mode product
  • Definition 3.2: Einstein product
  • Definition 3.3
  • Remark 1
  • Definition 3.4
  • Definition 3.5
  • Proposition 3.6
  • Proposition 3.7
  • Proposition 3.8
  • Proposition 3.9
  • ...and 15 more