Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

dCMF: Learning interpretable evolving patterns from temporal multiway data

Christos Chatzis, Carla Schenker, Jérémy E. Cohen, Evrim Acar

TL;DR

This paper addresses learning evolving patterns in temporal multiway data by bridging tensor factorization and dynamical modeling. It introduces dCMF, a coupled matrix factorization with a time-aware linear dynamical system (LDS) constraint that guides latent factors across time. Key contributions include formulating the LDS-based regularization, deriving an AO-ADMM optimization, and demonstrating through synthetic experiments that dCMF matches PARAFAC2 on PARAFAC2-structured data but outperforms when dynamics violate PARAFAC2; it also shows gains when prior transition information is available. The work highlights improved uniqueness properties and potential for capturing more complex temporal dynamics, with future directions including learning the transition matrix.

Abstract

Multiway datasets are commonly analyzed using unsupervised matrix and tensor factorization methods to reveal underlying patterns. Frequently, such datasets include timestamps and could correspond to, for example, health-related measurements of subjects collected over time. The temporal dimension is inherently different from the other dimensions, requiring methods that account for its intrinsic properties. Linear Dynamical Systems (LDS) are specifically designed to capture sequential dependencies in the observed data. In this work, we bridge the gap between tensor factorizations and dynamical modeling by exploring the relationship between LDS, Coupled Matrix Factorizations (CMF) and the PARAFAC2 model. We propose a time-aware coupled factorization model called d(ynamical)CMF that constrains the temporal evolution of the latent factors to adhere to a specific LDS structure. Using synthetic datasets, we compare the performance of dCMF with PARAFAC2 and t(emporal)PARAFAC2 which incorporates temporal smoothness. Our results show that dCMF and PARAFAC2-based approaches perform similarly when capturing smoothly evolving patterns that adhere to the PARAFAC2 structure. However, dCMF outperforms alternatives when the patterns evolve smoothly but deviate from the PARAFAC2 structure. Furthermore, we demonstrate that the proposed dCMF method enables to capture more complex dynamics when additional prior information about the temporal evolution is incorporated.

dCMF: Learning interpretable evolving patterns from temporal multiway data

TL;DR

This paper addresses learning evolving patterns in temporal multiway data by bridging tensor factorization and dynamical modeling. It introduces dCMF, a coupled matrix factorization with a time-aware linear dynamical system (LDS) constraint that guides latent factors across time. Key contributions include formulating the LDS-based regularization, deriving an AO-ADMM optimization, and demonstrating through synthetic experiments that dCMF matches PARAFAC2 on PARAFAC2-structured data but outperforms when dynamics violate PARAFAC2; it also shows gains when prior transition information is available. The work highlights improved uniqueness properties and potential for capturing more complex temporal dynamics, with future directions including learning the transition matrix.

Abstract

Multiway datasets are commonly analyzed using unsupervised matrix and tensor factorization methods to reveal underlying patterns. Frequently, such datasets include timestamps and could correspond to, for example, health-related measurements of subjects collected over time. The temporal dimension is inherently different from the other dimensions, requiring methods that account for its intrinsic properties. Linear Dynamical Systems (LDS) are specifically designed to capture sequential dependencies in the observed data. In this work, we bridge the gap between tensor factorizations and dynamical modeling by exploring the relationship between LDS, Coupled Matrix Factorizations (CMF) and the PARAFAC2 model. We propose a time-aware coupled factorization model called d(ynamical)CMF that constrains the temporal evolution of the latent factors to adhere to a specific LDS structure. Using synthetic datasets, we compare the performance of dCMF with PARAFAC2 and t(emporal)PARAFAC2 which incorporates temporal smoothness. Our results show that dCMF and PARAFAC2-based approaches perform similarly when capturing smoothly evolving patterns that adhere to the PARAFAC2 structure. However, dCMF outperforms alternatives when the patterns evolve smoothly but deviate from the PARAFAC2 structure. Furthermore, we demonstrate that the proposed dCMF method enables to capture more complex dynamics when additional prior information about the temporal evolution is incorporated.

Paper Structure

This paper contains 12 sections, 10 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Background
  3. Proposed method: d(ynamical)CMF
  4. The LDS constraint
  5. Optimization
  6. Experimental evaluation
  7. Experimental setup
  8. Smooth PARAFAC2 data
  9. Non-PARAFAC2 smooth data
  10. Non-identity transition matrix
  11. Uniqueness
  12. Conclusion

Figures (3)

  • Figure 1: The dCMF model. Evolving patterns captured by the columns of factor matrix ${\bm{\mathbf{B}}}_k$ are related to the previous ${\bm{\mathbf{B}}}_{k-1}$ through the transition matrix ${\bm{\mathbf{H}}}$.
  • Figure 2: Accuracy of methods in terms of FMS in each experiment. Each boxplot contains 30 points, one for the best run of each method for each dataset. Multiple regularization strengths are considered for temporal smoothness (tPARAFAC2) and the proposed LDS constraint (dCMF). PAR2 is short for PARAFAC2.
  • Figure 3: Boxplots show the FMS of all runs when compared with the best run for each dataset (at noise level $\eta=0.75$). For each experiment, there are in total 570 runs (30 datasets $\times$ 20 initializations $-$ the best run for each dataset). $n$ denotes the number of runs after discarding failed runs.