Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Tensor Decomposition Meets RKHS: Efficient Algorithms for Smooth and Misaligned Data

Brett W. Larsen, Tamara G. Kolda, Anru R. Zhang, Alex H. Williams

TL;DR

CP-HiFi does not require the observed data to lie on a regular and finite rectangular grid and naturally incorporates misaligned data, and can enforce smoothness in the infinite dimensional modes.

Abstract

The canonical polyadic (CP) tensor decomposition decomposes a multidimensional data array into a sum of outer products of finite-dimensional vectors. Instead, we can replace some or all of the vectors with continuous functions (infinite-dimensional vectors) from a reproducing kernel Hilbert space (RKHS). We refer to tensors with some infinite-dimensional modes as quasitensors, and the approach of decomposing a tensor with some continuous RKHS modes is referred to as CP-HiFi (hybrid infinite and finite dimensional) tensor decomposition. An advantage of CP-HiFi is that it can enforce smoothness in the infinite dimensional modes. Further, CP-HiFi does not require the observed data to lie on a regular and finite rectangular grid and naturally incorporates misaligned data. We detail the methodology and illustrate it on a synthetic example.

Tensor Decomposition Meets RKHS: Efficient Algorithms for Smooth and Misaligned Data

TL;DR

CP-HiFi does not require the observed data to lie on a regular and finite rectangular grid and naturally incorporates misaligned data, and can enforce smoothness in the infinite dimensional modes.

Abstract

The canonical polyadic (CP) tensor decomposition decomposes a multidimensional data array into a sum of outer products of finite-dimensional vectors. Instead, we can replace some or all of the vectors with continuous functions (infinite-dimensional vectors) from a reproducing kernel Hilbert space (RKHS). We refer to tensors with some infinite-dimensional modes as quasitensors, and the approach of decomposing a tensor with some continuous RKHS modes is referred to as CP-HiFi (hybrid infinite and finite dimensional) tensor decomposition. An advantage of CP-HiFi is that it can enforce smoothness in the infinite dimensional modes. Further, CP-HiFi does not require the observed data to lie on a regular and finite rectangular grid and naturally incorporates misaligned data. We detail the methodology and illustrate it on a synthetic example.
Paper Structure (44 sections, 3 theorems, 82 equations, 6 figures, 1 table, 2 algorithms)

This paper contains 44 sections, 3 theorems, 82 equations, 6 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries: Matrices and tensors
  3. Matrix operations
  4. Weighted norm
  5. Hadamard product
  6. Kronecker product
  7. Khatri-Rao product
  8. Matrix calculus
  9. Quasimatrices
  10. Tensor operations and tensor decompositions
  11. Vectorization
  12. Mode-$k$ Unfolding
  13. MTTKRP
  14. Norms
  15. Preliminaries: Reproducing kernel Hilbert spaces (RKHS)
  16. ...and 29 more sections

Key Result

Theorem 3.3

For a positive semi-definite kernel $\bm{\mathcal{K}}$, its associated RKHS $\mathcal{H_{\bm{\mathcal{K}}}}$, and a set of $n$ observations $\set{(x_i, y_i)}_{i=1}^p$, consider the regularized empirical risk minimization problem defined by the loss function $\bm{{\mathit{L}}} {^{\intercal}}{ { where $\lambda > 0$. The minimizer of this problem $\bm{{\mathnormal{f}}} {^{\intercal}}{ {^{

Figures (6)

  • Figure 1: Example $4 \times 3 \times [0,1]$ quasitensor: $\boldsymbol{{\mathcal{T}}} (i,j,x)$.
  • Figure 1: (Experiment 1) 30 aligned points
  • Figure 2: (Experiment 2) 12 aligned points
  • Figure 3: (Experiment 3) 12 dense aligned samples for each $\boldsymbol{{\mathcal{T}}} _{ij}$ plus one extra data point for $\boldsymbol{{\mathcal{T}}} _{22}$ at $x = 0.7677$
  • Figure 4: (Experiment 4) 12 dense aligned samples for each $\boldsymbol{{\mathcal{T}}} _{ij}$ plus one extra data point for each $\boldsymbol{{\mathcal{T}}} _{ij}$
  • ...and 1 more figures

Theorems & Definitions (4)

  • Theorem 3.3: Representer Theorem
  • Theorem 3.4: Mercer's Theorem
  • Corollary 3.5
  • Definition 3.6: Universal Kernel MiXuZh06