Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Learning For Time Series Analysis With Application On Human Motion

Ali Ismail-Fawaz

TL;DR

The thesis addresses the bottlenecks of time series analysis by advancing deep learning approaches for classification, clustering, and regression, with a focus on human motion data. It introduces foundation-model-inspired strategies, including hand-crafted convolution filters and pre-training (PHIT), to improve generalization across diverse TS domains while reducing model complexity through LITE/LITEMV architectures. The work also emphasizes semi-supervised/self-supervised learning (TRILITE) to cope with limited labels and proposes ShapeDBA-based prototyping and SVAE-based generative models to augment data and enable realistic motion synthesis. A major contribution is the development of a reproducible open-source ecosystem (aeon/aeonaeon-paper) and a robust evaluation framework (MCM) for discriminative and generative TS models, including a unified metric suite for human motion generation. Collectively, the contributions push practical time series analysis forward, enabling scalable, efficient, and explainable human motion understanding in medical, cinematic, and industrial settings, while advancing foundational methods for cross-domain TS tasks.

Abstract

Time series data, defined by equally spaced points over time, is essential in fields like medicine, telecommunications, and energy. Analyzing it involves tasks such as classification, clustering, prototyping, and regression. Classification identifies normal vs. abnormal movements in skeleton-based motion sequences, clustering detects stock market behavior patterns, prototyping expands physical therapy datasets, and regression predicts patient recovery. Deep learning has recently gained traction in time series analysis due to its success in other domains. This thesis leverages deep learning to enhance classification with feature engineering, introduce foundation models, and develop a compact yet state-of-the-art architecture. We also address limited labeled data with self-supervised learning. Our contributions apply to real-world tasks, including human motion analysis for action recognition and rehabilitation. We introduce a generative model for human motion data, valuable for cinematic production and gaming. For prototyping, we propose a shape-based synthetic sample generation method to support regression models when data is scarce. Lastly, we critically evaluate discriminative and generative models, identifying limitations in current methodologies and advocating for a robust, standardized evaluation framework. Our experiments on public datasets provide novel insights and methodologies, advancing time series analysis with practical applications.

Deep Learning For Time Series Analysis With Application On Human Motion

TL;DR

The thesis addresses the bottlenecks of time series analysis by advancing deep learning approaches for classification, clustering, and regression, with a focus on human motion data. It introduces foundation-model-inspired strategies, including hand-crafted convolution filters and pre-training (PHIT), to improve generalization across diverse TS domains while reducing model complexity through LITE/LITEMV architectures. The work also emphasizes semi-supervised/self-supervised learning (TRILITE) to cope with limited labels and proposes ShapeDBA-based prototyping and SVAE-based generative models to augment data and enable realistic motion synthesis. A major contribution is the development of a reproducible open-source ecosystem (aeon/aeonaeon-paper) and a robust evaluation framework (MCM) for discriminative and generative TS models, including a unified metric suite for human motion generation. Collectively, the contributions push practical time series analysis forward, enabling scalable, efficient, and explainable human motion understanding in medical, cinematic, and industrial settings, while advancing foundational methods for cross-domain TS tasks.

Abstract

Time series data, defined by equally spaced points over time, is essential in fields like medicine, telecommunications, and energy. Analyzing it involves tasks such as classification, clustering, prototyping, and regression. Classification identifies normal vs. abnormal movements in skeleton-based motion sequences, clustering detects stock market behavior patterns, prototyping expands physical therapy datasets, and regression predicts patient recovery. Deep learning has recently gained traction in time series analysis due to its success in other domains. This thesis leverages deep learning to enhance classification with feature engineering, introduce foundation models, and develop a compact yet state-of-the-art architecture. We also address limited labeled data with self-supervised learning. Our contributions apply to real-world tasks, including human motion analysis for action recognition and rehabilitation. We introduce a generative model for human motion data, valuable for cinematic production and gaming. For prototyping, we propose a shape-based synthetic sample generation method to support regression models when data is scarce. Lastly, we critically evaluate discriminative and generative models, identifying limitations in current methodologies and advocating for a robust, standardized evaluation framework. Our experiments on public datasets provide novel insights and methodologies, advancing time series analysis with practical applications.

Paper Structure

This paper contains 223 sections, 5 theorems, 118 equations, 108 figures, 14 tables, 9 algorithms.

Table of Contents

  1. State Of The Art For Time Series Analysis: Supervised and Unsupervised Learning
  2. Introduction
  3. Supervised Learning: Time Series Classification and Extrinsic Regression
  4. Time Series Classification
  5. Distance Based Methods
  6. $k$-Nearest Neighbor - Dynamic Time Warping ($k$-NN-DTW) and Variants
  7. Elastic Ensemble
  8. Proximity Forest and Proximity Forest2.0
  9. Feature Based Methods
  10. The Canonical Time Series Characteristics (Catch22)
  11. Time Series Feature Extraction based on Scalable Hypothesis Tests (TSFresh) and The FreshPRINCE
  12. Convolutional Based Methods
  13. RandOm Convolutional KErnel Transform (ROCKET)
  14. MiniROCKET & MultiROCKET
  15. HYbrid Dictionary-Rocket Architecture
  16. ...and 208 more sections

Key Result

Theorem 1

Let $K$ be an even positive integer, a convolutional filter $\textbf{w}_{I_K}=[(-1)^{k}$ for $k \in \{1,...,K\}]$ is an increasing trend detection filter of time series, i.e. it only activates (produces positive values) on increasing trend segments.

Figures (108)

  • Figure 1: Time Series Extrinsic Regression (TSER) is the task of predicting continuous labels of the time series samples.
  • Figure 2: Time Series CLustering (TSCL) is the task of discovering common information between samples of time series in order to group them into clusters.
  • Figure 3: Time Series Prototyping (TSP) is the task of finding a representative of a collection of time series of a similar group.
  • Figure 4: Time Series Classification (TSC) is the task of predicting a discrete label of the time series samples.
  • Figure 5: The number of research papers mentioning "deep learning" and "time series classification" increased rapidly in the last years.
  • ...and 103 more figures

Theorems & Definitions (5)

  • Theorem 1: Increasing Trend Dection Convolution Filter
  • Theorem 2: Decreasing Trend Dection Convolution Filter
  • Theorem 3: Fréchet Inception Distance Interpretation
  • Theorem 4: Average Pair Distance Interpretation
  • Theorem 5: Warping Path Diversity's Distance To Diaginal Computation