Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Convergence of the alternating least squares algorithm for CP tensor decompositions

Nicholas Hu, Mark A. Iwen, Deanna Needell, Rongrong Wang

TL;DR

It is shown that CP-AltLS converges polynomially with order $N-1$ for $N$th-order orthogonally decomposable tensors and linearly for incoherently decomposable tensors, with convergence being measured in terms of the angles between the factors of the exact tensor and those of the approximate tensor.

Abstract

The alternating least squares (ALS/AltLS) method is a widely used algorithm for computing the CP decomposition of a tensor. However, its convergence theory is still incompletely understood. In this paper, we prove explicit quantitative local convergence theorems for CP-AltLS applied to orthogonally decomposable and incoherently decomposable tensors. Specifically, we show that CP-AltLS converges polynomially with order $N-1$ for $N$th-order orthogonally decomposable tensors and linearly for incoherently decomposable tensors, with convergence being measured in terms of the angles between the factors of the exact tensor and those of the approximate tensor. Unlike existing results, our analysis is both quantitative and constructive, applying to standard CP-AltLS and accommodating factor matrices with small but nonzero mutual coherence, while remaining applicable to tensors of arbitrary rank. We also confirm these rates of convergence numerically and investigate accelerating the convergence of CP-AltLS using an SVD-based coherence reduction scheme.

Convergence of the alternating least squares algorithm for CP tensor decompositions

TL;DR

It is shown that CP-AltLS converges polynomially with order for th-order orthogonally decomposable tensors and linearly for incoherently decomposable tensors, with convergence being measured in terms of the angles between the factors of the exact tensor and those of the approximate tensor.

Abstract

The alternating least squares (ALS/AltLS) method is a widely used algorithm for computing the CP decomposition of a tensor. However, its convergence theory is still incompletely understood. In this paper, we prove explicit quantitative local convergence theorems for CP-AltLS applied to orthogonally decomposable and incoherently decomposable tensors. Specifically, we show that CP-AltLS converges polynomially with order for th-order orthogonally decomposable tensors and linearly for incoherently decomposable tensors, with convergence being measured in terms of the angles between the factors of the exact tensor and those of the approximate tensor. Unlike existing results, our analysis is both quantitative and constructive, applying to standard CP-AltLS and accommodating factor matrices with small but nonzero mutual coherence, while remaining applicable to tensors of arbitrary rank. We also confirm these rates of convergence numerically and investigate accelerating the convergence of CP-AltLS using an SVD-based coherence reduction scheme.

Paper Structure

This paper contains 20 sections, 12 theorems, 66 equations, 4 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Prior work
  3. Contributions
  4. Organization
  5. Notation and preliminaries
  6. Tensor operations
  7. Tensor CP decompositions
  8. Local convergence of CP-AltLS
  9. Local convergence of CP-AltLS for orthogonally decomposable tensors
  10. Local convergence of CP-AltLS for incoherently decomposable tensors
  11. Local convergence of CP-SAltLS
  12. Global convergence of CP-AltLS
  13. Experiments
  14. Orthogonally decomposable tensors
  15. Incoherently decomposable tensors
  16. ...and 5 more sections

Key Result

Theorem 3.2

Suppose that $\bm{\mathscr{X}} \in \mathbb{R}^{I_1 \times \cdots \times I_N}$($N \geq 3$) is orthogonally decomposable and write $\bm{\mathscr{X}} = \llbracket \bm{\mathrm{\lambda}} \,;\, \bm{\mathrm{A}}^{(1)}_{} , \dots, \bm{\mathrm{A}}^{(N)}_{} \rrbracket$ for some $\bm{\mathrm{\lambda}} \i and $\kappa \vcentcolon= {\max_{r \in [R]} \lvert \lambda_r \rvert} / {\min_{r \in [R]} \lvert \lam

Figures (4)

  • Figure 1: Polynomial convergence of \ref{['CP-AltLS']} for $N$th-order orthogonally decomposable tensors.
  • Figure 2: Linear convergence of \ref{['CP-AltLS']} for $N$th-order incoherently decomposable tensors.
  • Figure 3: Convergence of weights in \ref{['CP-AltLS']} for 3rd-order orthogonally and incoherently decomposable tensors.
  • Figure 4: Convergence of \ref{['CP-SAltLS']} for coherence reduction parameter values ranging from $\omega = 1$ (no coherence reduction) to $\omega = 0$ (complete coherence reduction).

Theorems & Definitions (34)

  • Definition 2.1: Inner product and norm of tensors
  • Definition 2.2: $(p, q)$-norm of a matrix
  • Definition 2.3: Fibres and matricization
  • Definition 2.4: Diagonal and off-diagonal parts of a matrix
  • Definition 2.5: CP decomposition and rank of a tensor
  • Definition 2.6: Orthogonally decomposable tensor
  • Remark 2.7
  • Definition 2.8: Coherence
  • Definition 2.9: Incoherently decomposable tensor
  • Remark 3.1
  • ...and 24 more