Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enabling Tensor Decomposition for Time-Series Classification via A Simple Pseudo-Laplacian Contrast

Man Li, Ziyue Li, Lijun Sun, Fugee Tsung

TL;DR

This work tackles the challenge that tensor decomposition, though effective for reconstruction, often underperforms in classification due to non-uniqueness and rotation invariance of factorization. It introduces Pseudo-Laplacian Contrast (PLC), a cross-view graph Laplacian framework that learns a pseudo graph from latent features and augments them with class-preserving data transformations to produce class-aware representations within a CP-based tensor model. The method unifies pseudo-graph learning, cross-view Laplacian regularization, and reconstruction through an unsupervised ALS optimization, linking closely to contrastive learning with a block-contrastive structure. Empirical results on HAR, Sleep-EDF, and PTB-XL demonstrate improved classification accuracy and robust generalization, with pseudo graphs that align closely to ground-truth class structure, highlighting PLC’s potential for efficient time-series classification without heavy supervision.

Abstract

Tensor decomposition has emerged as a prominent technique to learn low-dimensional representation under the supervision of reconstruction error, primarily benefiting data inference tasks like completion and imputation, but not classification task. We argue that the non-uniqueness and rotation invariance of tensor decomposition allow us to identify the directions with largest class-variability and simple graph Laplacian can effectively achieve this objective. Therefore we propose a novel Pseudo Laplacian Contrast (PLC) tensor decomposition framework, which integrates the data augmentation and cross-view Laplacian to enable the extraction of class-aware representations while effectively capturing the intrinsic low-rank structure within reconstruction constraint. An unsupervised alternative optimization algorithm is further developed to iteratively estimate the pseudo graph and minimize the loss using Alternating Least Square (ALS). Extensive experimental results on various datasets demonstrate the effectiveness of our approach.

Enabling Tensor Decomposition for Time-Series Classification via A Simple Pseudo-Laplacian Contrast

TL;DR

This work tackles the challenge that tensor decomposition, though effective for reconstruction, often underperforms in classification due to non-uniqueness and rotation invariance of factorization. It introduces Pseudo-Laplacian Contrast (PLC), a cross-view graph Laplacian framework that learns a pseudo graph from latent features and augments them with class-preserving data transformations to produce class-aware representations within a CP-based tensor model. The method unifies pseudo-graph learning, cross-view Laplacian regularization, and reconstruction through an unsupervised ALS optimization, linking closely to contrastive learning with a block-contrastive structure. Empirical results on HAR, Sleep-EDF, and PTB-XL demonstrate improved classification accuracy and robust generalization, with pseudo graphs that align closely to ground-truth class structure, highlighting PLC’s potential for efficient time-series classification without heavy supervision.

Abstract

Tensor decomposition has emerged as a prominent technique to learn low-dimensional representation under the supervision of reconstruction error, primarily benefiting data inference tasks like completion and imputation, but not classification task. We argue that the non-uniqueness and rotation invariance of tensor decomposition allow us to identify the directions with largest class-variability and simple graph Laplacian can effectively achieve this objective. Therefore we propose a novel Pseudo Laplacian Contrast (PLC) tensor decomposition framework, which integrates the data augmentation and cross-view Laplacian to enable the extraction of class-aware representations while effectively capturing the intrinsic low-rank structure within reconstruction constraint. An unsupervised alternative optimization algorithm is further developed to iteratively estimate the pseudo graph and minimize the loss using Alternating Least Square (ALS). Extensive experimental results on various datasets demonstrate the effectiveness of our approach.
Paper Structure (16 sections, 2 theorems, 15 equations, 7 figures, 1 table)

This paper contains 16 sections, 2 theorems, 15 equations, 7 figures, 1 table.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Notations.
  4. Tensor modeling.
  5. Problem formulation.
  6. Methodology
  7. PLC framework
  8. To learn a pseudo signed graph
  9. To enhance the feature via pseudo-graph Laplacian
  10. Deep Dive: Relationship with contrastive learning
  11. Unsupervised alternative optimization
  12. Experiments
  13. Experimental setup
  14. Datasets.
  15. Result analysis
  16. ...and 1 more sections

Key Result

Proposition 1

Linear Convergence of $\mathbf{w}^{(n)}$Given non-zero vectors $\mathbf{H}_{1}$, $\mathbf{H}_{2}$, $\mathbf{w}^{(n)}_{0}$, and $\alpha, \beta >0$, we have ${\left\|\mathbf{w}^{(n)}_{t+1}-\mathbf{w}^{(n)^*}\right\|_{2}}\leq \frac{\beta (2m+M){\left\|\mathbf{H}_{2}\right\|_{2}}{\left\|\mathbf{H}_{1}^{

Figures (7)

  • Figure 1: The class information in the feature space: (a) the features extracted by CP decomposition are messed together; (b) instance-wise contrast (e.g., ATD y:22 and InfoNCE oord2018representation) treating the data from the same instances as a positive pair only presents obscure class patterns; (c) cross-view Laplacian iteratively learn pseudo graph and use cross-view Laplacian to enhance the class-distinctiveness. ( $\dashleftarrow\dashrightarrow$: pull far away, $\rightarrow\leftarrow$: push closer with darker color indicating more impact).
  • Figure 2: (a) Original $\mathcal{X}$ is augmented to $\widetilde{\mathcal{X}}$; both are decomposed into shared matrices $\mathbf{A,B,C}$ and their own feature vectors $\mathbf{w}, \widetilde{\mathbf{w}}$. An off-shelf clustering groups the features and gets a pseudo-signed graph indicating class relations. The pseudo-Laplacian contrast pushes the cross-view features from the same pseudo-class (colored) together and different classes away; The transformation invariance (need no pseudo-class, grey) only pushes the same sample together and different samples away; (b) When reconstruction converges in a small region, decomposition results are non-identifiable; thanks to the rotation-invariance, our PLC finds a best rotating angle to best present the class information.
  • Figure 3: The design of PLC loss.
  • Figure 4: (a) The classification performance over varying training size for classifier: to achieve same accuracy, ours can need 3 times less data than ATD; (b) The curve of reconstruction loss and classification loss: the two term improve each other and classification accuracy keeps increasing after the convergence of reconstruction loss; (c) Supervised contrast with true label can be seen as the upper bound of our pseudo-graph, and our pseudo-Laplacian contrast has only a very tiny gap with the upper bound. (d) Ablation study of "w/o PLC" (no $\ell_{\textit{PLC}}$), "w/o cross-view" (no $\ell_{\textit{cross-view}}$) and "w/o trans-inv" (no $\ell_{\textit{trans-inv}}$).
  • Figure 5: The visualization of $\mathbf{w}^{(n)}$ and pseudo graph $\mathcal{G}$ on HAR dataset: (a) the t-SNE of the learned feature at the beginning and convergence where our model learns more clearly-separated class embedding; (b) the adjacent matrix of the pseudo-graph is progressively approaching that of the true graph.
  • ...and 2 more figures

Theorems & Definitions (2)

  • Proposition 1
  • Proposition 2