Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TT-LSQR For Tensor Least Squares Problems and Application to Data Mining *

Lorenzo Piccinini, Valeria Simoncini

TL;DR

The paper addresses solving multiterm tensor least squares problems in which the unknown tensor is stored in Tensor-Train format, leveraging a tensorized LSQR (TT-LSQR) method to avoid explicit vectorization. It introduces TT-LSQR with rank-truncation (TT-SVD rounding) to keep TT-ranks small and preserves structure throughout iterations, and augments the approach with Johnson–Lindenstrauss sketching and problem-specific preconditioning to reduce cost and improve conditioning. The authors demonstrate the method on discretized PDEs to verify numerical stability and on information retrieval tasks to allocate a new query into clustered document groups, showing competitive accuracy and favorable memory usage compared with matrix-based approaches. The work highlights the practical viability of multiterm tensor least squares for large-scale, multi-way data and provides a scalable framework that can be extended with more datasets, refined sketches, and advanced truncation strategies.

Abstract

We are interested in the numerical solution of the tensor least squares problem \[ \min_{\mathcal{X}} \| \mathcal{F} - \sum_{i =1}^{\ell} \mathcal{X} \times_1 A_1^{(i)} \times_2 A_2^{(i)} \cdots \times_d A_d^{(i)} \|_F, \] where $\mathcal{X}\in\mathbb{R}^{m_1 \times m_2 \times \cdots \times m_d}$, $\mathcal{F}\in\mathbb{R}^{n_1\times n_2 \times \cdots \times n_d}$ are tensors with $d$ dimensions, and the coefficients $A_j^{(i)}$ are tall matrices of conforming dimensions. We first describe a tensor implementation of the classical LSQR method by Paige and Saunders, using the tensor-train representation as key ingredient. We also show how to incorporate sketching to lower the computational cost of dealing with the tall matrices $A_j^{(i)}$. We then use this methodology to address a problem in information retrieval, the classification of a new query document among already categorized documents, according to given keywords.

TT-LSQR For Tensor Least Squares Problems and Application to Data Mining *

TL;DR

The paper addresses solving multiterm tensor least squares problems in which the unknown tensor is stored in Tensor-Train format, leveraging a tensorized LSQR (TT-LSQR) method to avoid explicit vectorization. It introduces TT-LSQR with rank-truncation (TT-SVD rounding) to keep TT-ranks small and preserves structure throughout iterations, and augments the approach with Johnson–Lindenstrauss sketching and problem-specific preconditioning to reduce cost and improve conditioning. The authors demonstrate the method on discretized PDEs to verify numerical stability and on information retrieval tasks to allocate a new query into clustered document groups, showing competitive accuracy and favorable memory usage compared with matrix-based approaches. The work highlights the practical viability of multiterm tensor least squares for large-scale, multi-way data and provides a scalable framework that can be extended with more datasets, refined sketches, and advanced truncation strategies.

Abstract

We are interested in the numerical solution of the tensor least squares problem where , are tensors with dimensions, and the coefficients are tall matrices of conforming dimensions. We first describe a tensor implementation of the classical LSQR method by Paige and Saunders, using the tensor-train representation as key ingredient. We also show how to incorporate sketching to lower the computational cost of dealing with the tall matrices . We then use this methodology to address a problem in information retrieval, the classification of a new query document among already categorized documents, according to given keywords.

Paper Structure

This paper contains 15 sections, 2 theorems, 36 equations, 7 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Truncated Tensor-Oriented LSQR
  4. Tensor-train LSQR
  5. Computational devices to improve performance
  6. Preconditioning for Tensor-oriented LSQR
  7. Sketching for performance
  8. Computational experiments with discretized PDEs
  9. Information retrieval, discrimination and classification in data mining
  10. Discriminant analysis on clustered data
  11. Computational experiments: Preparing the setting
  12. Computational experiments: Testing the collocation strategy
  13. Computational experiments with sketching
  14. Conclusions
  15. Acknowledgments

Key Result

Theorem 3.1

(OSELEDETS201070). Suppose that the unfoldings $A_k$ of the tensor $\mathcal{A}$ satisfy (lowrank). The TT-SVD computes a tensor $\mathcal{B}$ in the TT-format with TT-ranks $r_k$ and where $\epsilon_k$ is the distance (in the Frobenius norm) from $A_k$ to its best rank-$r_k$ approximation

Figures (7)

  • Figure 1: Tucker format.
  • Figure 1: Convergence history of TT-LSQR for $n=50$ for problem (\ref{['eqn:pde_eq']}). Left: Behavior as the TT truncation threshold varies. Right: Behavior as the TT truncation rank varies (truncation threshold $10^{-8}$). \newlabelfig:pde_param0
  • Figure 1: Sparsity pattern of correlation matrices of ${\@fontswitch{}{\mathcal{}} A}$ in (\ref{['eqn:calA']}) with $m=60$ for Reuters(left), Cranfield(middle), Medline (right); reported are elements above 0.15 in absolute value. \newlabelfig:spydata0
  • Figure 1: Tensor core of the sum of two Tucker tensors, $\mathcal{S}={\rm blkdiag}(\mathcal{H},\mathcal{G})$.
  • Figure 2: Tensor-Train format.
  • ...and 2 more figures

Theorems & Definitions (3)

  • Definition 2.1
  • Theorem 3.1
  • Theorem 9.1