Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Improving Personalisation in Valence and Arousal Prediction using Data Augmentation

Munachiso Nwadike, Jialin Li, Hanan Salam

TL;DR

This work tackles the challenge of personalisation in continuous valence and arousal prediction under limited target data. It introduces Distance Weighting Augmentation (DWA), a data-level augmentation method that uses segment-level similarity from a global pool to augment the target individual's data, in combination with transfer learning. On the MuSe-Personalisation Ulm-TSST dataset, DWA improves performance for features with low baselines and achieves a peak combined CCC of 0.78, with arousal/valence peaks of 0.81 and 0.76, respectively. These results demonstrate the practical value of data-level personalisation for multimodal affective computing and provide guidance on deploying personalised models in real-world HMI contexts.

Abstract

In the field of emotion recognition and Human-Machine Interaction (HMI), personalised approaches have exhibited their efficacy in capturing individual-specific characteristics and enhancing affective prediction accuracy. However, personalisation techniques often face the challenge of limited data for target individuals. This paper presents our work on an enhanced personalisation strategy, that leverages data augmentation to develop tailored models for continuous valence and arousal prediction. Our proposed approach, Distance Weighting Augmentation (DWA), employs a weighting-based augmentation method that expands a target individual's dataset, leveraging distance metrics to identify similar samples at the segment-level. Experimental results on the MuSe-Personalisation 2023 Challenge dataset demonstrate that our method significantly improves the performance of features sets which have low baseline performance, on the test set. This improvement in poor-performing features comes without sacrificing performance on high-performing features. In particular, our method achieves a maximum combined testing CCC of 0.78, compared to the reported baseline score of 0.76 (reproduced at 0.72). It also achieved a peak arousal and valence scores of 0.81 and 0.76, compared to reproduced baseline scores of 0.76 and 0.67 respectively. Through this work, we make significant contributions to the advancement of personalised affective computing models, enhancing the practicality and adaptability of data-level personalisation in real world contexts.

Improving Personalisation in Valence and Arousal Prediction using Data Augmentation

TL;DR

This work tackles the challenge of personalisation in continuous valence and arousal prediction under limited target data. It introduces Distance Weighting Augmentation (DWA), a data-level augmentation method that uses segment-level similarity from a global pool to augment the target individual's data, in combination with transfer learning. On the MuSe-Personalisation Ulm-TSST dataset, DWA improves performance for features with low baselines and achieves a peak combined CCC of 0.78, with arousal/valence peaks of 0.81 and 0.76, respectively. These results demonstrate the practical value of data-level personalisation for multimodal affective computing and provide guidance on deploying personalised models in real-world HMI contexts.

Abstract

In the field of emotion recognition and Human-Machine Interaction (HMI), personalised approaches have exhibited their efficacy in capturing individual-specific characteristics and enhancing affective prediction accuracy. However, personalisation techniques often face the challenge of limited data for target individuals. This paper presents our work on an enhanced personalisation strategy, that leverages data augmentation to develop tailored models for continuous valence and arousal prediction. Our proposed approach, Distance Weighting Augmentation (DWA), employs a weighting-based augmentation method that expands a target individual's dataset, leveraging distance metrics to identify similar samples at the segment-level. Experimental results on the MuSe-Personalisation 2023 Challenge dataset demonstrate that our method significantly improves the performance of features sets which have low baseline performance, on the test set. This improvement in poor-performing features comes without sacrificing performance on high-performing features. In particular, our method achieves a maximum combined testing CCC of 0.78, compared to the reported baseline score of 0.76 (reproduced at 0.72). It also achieved a peak arousal and valence scores of 0.81 and 0.76, compared to reproduced baseline scores of 0.76 and 0.67 respectively. Through this work, we make significant contributions to the advancement of personalised affective computing models, enhancing the practicality and adaptability of data-level personalisation in real world contexts.
Paper Structure (21 sections, 5 equations, 2 figures, 6 tables, 1 algorithm)

This paper contains 21 sections, 5 equations, 2 figures, 6 tables, 1 algorithm.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORK
  3. Generic Multimodal Emotion Recognition
  4. Personalised Multi-modal Emotion Recognition
  5. Individual Model Transfer Learning
  6. Sample Re-weighting
  7. PROPOSED METHODOLOGY
  8. Distance Weighting Augmentation
  9. Distance Metrics
  10. Centroid L2 Distance
  11. Centroid Dot Product (centroid DP) Distance
  12. Cosine Distance
  13. EXPERIMENTAL SETUP
  14. Baseline Models
  15. Dataset
  16. ...and 6 more sections

Figures (2)

  • Figure 1: Our personalisation framework leverages transfer learning together with distance weighting augmentation.
  • Figure 2: DWA generates an dataset $D_{I_i{aug}}$ from the original dataset $D_{I_i}$, by generating an augmentation pool, and selecting most similar samples from that pool, for target segment $W_T$.