Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TraM : Enhancing User Sleep Prediction with Transformer-based Multivariate Time Series Modeling and Machine Learning Ensembles

Jinjae Kim, Minjeong Ma, Eunjee Choi, Keunhee Cho, Chanwoo Lee

Abstract

This paper presents a novel approach that leverages Transformer-based multivariate time series model and Machine Learning Ensembles to predict the quality of human sleep, emotional states, and stress levels. A formula to calculate the labels was developed, and the various models were applied to user data. Time Series Transformer was used for labels where time series characteristics are crucial, while Machine Learning Ensembles were employed for labels requiring comprehensive daily activity statistics. Time Series Transformer excels in capturing the characteristics of time series through pre-training, while Machine Learning Ensembles select machine learning models that meet our categorization criteria. The proposed model, TraM, scored 6.10 out of 10 in experiments, demonstrating superior performance compared to other methodologies. The code and configuration for the TraM framework are available at: https://github.com/jin-jae/ETRI-Paper-Contest.

TraM : Enhancing User Sleep Prediction with Transformer-based Multivariate Time Series Modeling and Machine Learning Ensembles

Abstract

This paper presents a novel approach that leverages Transformer-based multivariate time series model and Machine Learning Ensembles to predict the quality of human sleep, emotional states, and stress levels. A formula to calculate the labels was developed, and the various models were applied to user data. Time Series Transformer was used for labels where time series characteristics are crucial, while Machine Learning Ensembles were employed for labels requiring comprehensive daily activity statistics. Time Series Transformer excels in capturing the characteristics of time series through pre-training, while Machine Learning Ensembles select machine learning models that meet our categorization criteria. The proposed model, TraM, scored 6.10 out of 10 in experiments, demonstrating superior performance compared to other methodologies. The code and configuration for the TraM framework are available at: https://github.com/jin-jae/ETRI-Paper-Contest.

Paper Structure

This paper contains 16 sections, 2 equations, 2 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Proposed Methods
  4. Time Series Transformer
  5. Preprocessing
  6. Time Series Transformer Model
  7. Pre-training
  8. Fine-tuning
  9. Machine Learning Ensembles
  10. Preprocessing
  11. MultiOutputClassifier
  12. Experiments
  13. Experiment Setup
  14. Dataset
  15. Results
  16. ...and 1 more sections

Figures (2)

  • Figure 1: Overview of our method. Our framework has two main components: a TST and Machine Learning Ensembles. On the left, TST utilizes a transformer encoder combined with learnable positional encoding and multiple input encoding blocks to process time series data. On the right, Machine Learning Ensembles shows the combination of various model types, integrated through soft voting to enhance prediction accuracy. We then concatenate Q1-Q3 labels from TST and S1-S4 labels from Machine Learning Ensembles.
  • Figure 2: F1 Scores (Macro) on public test dataset