Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive Temporal Dynamics for Personalized Emotion Recognition: A Liquid Neural Network Approach

Anindya Bhattacharjee, Nittya Ananda Biswas, K. A. Shahriar, Adib Rahman

TL;DR

This work tackles the challenge of personalized emotion recognition from non-stationary physiological signals by introducing an attention-augmented Liquid Time-Constant Network (LTC) framework that jointly models EEG and peripheral signals. The approach uses CNN front-ends for raw EEG, LTC-based temporal dynamics with learnable time constants, and a self-attention mechanism to summarize temporal information, paired with an autoencoder-based cross-modal fusion that includes personality context. On PhyMER, it achieves 95.45% accuracy across seven discrete emotions, with near-perfect class separability (AUCs ≈ 0.99) and interpretable temporal signatures via attention and t-SNE visualizations. The results reveal distinct temporal neuron groups and emotion-specific attention patterns, demonstrating the method’s potential for efficient, edge-friendly, personalized affective computing while outlining avenues for subject-independent generalization and lighter fusion strategies.

Abstract

Emotion recognition from physiological signals remains challenging due to their non-stationary, noisy, and subject-dependent characteristics. This work presents, to the best of our knowledge, the first comprehensive application of liquid neural networks for EEG-based emotion recognition. The proposed multimodal framework combines convolutional feature extraction, liquid neural networks with learnable time constants, and attention-guided fusion to model temporal EEG dynamics with complementary peripheral physiological and personality features. Dedicated subnetworks are used to process EEG features and auxiliary modalities, and a shared autoencoder-based fusion module is used to learn discriminative latent representations before classification. Subject-dependent experiments conducted on the PhyMER dataset across seven emotional classes achieve an accuracy of 95.45%, surpassing previously reported results. Furthermore, temporal attention analysis provides interpretable insights into emotion-specific temporal relevance, and t-SNE visualizations demonstrate enhanced class separability, highlighting the effectiveness of the proposed approach. Finally, statistical analysis of temporal dynamics confirms that the network self-organizes into distinct functional groups with specialized fast and slow neurons, proving it independently tunes learnable time constants and memory dominance to effectively capture complex emotion artifacts.

Adaptive Temporal Dynamics for Personalized Emotion Recognition: A Liquid Neural Network Approach

TL;DR

This work tackles the challenge of personalized emotion recognition from non-stationary physiological signals by introducing an attention-augmented Liquid Time-Constant Network (LTC) framework that jointly models EEG and peripheral signals. The approach uses CNN front-ends for raw EEG, LTC-based temporal dynamics with learnable time constants, and a self-attention mechanism to summarize temporal information, paired with an autoencoder-based cross-modal fusion that includes personality context. On PhyMER, it achieves 95.45% accuracy across seven discrete emotions, with near-perfect class separability (AUCs ≈ 0.99) and interpretable temporal signatures via attention and t-SNE visualizations. The results reveal distinct temporal neuron groups and emotion-specific attention patterns, demonstrating the method’s potential for efficient, edge-friendly, personalized affective computing while outlining avenues for subject-independent generalization and lighter fusion strategies.

Abstract

Emotion recognition from physiological signals remains challenging due to their non-stationary, noisy, and subject-dependent characteristics. This work presents, to the best of our knowledge, the first comprehensive application of liquid neural networks for EEG-based emotion recognition. The proposed multimodal framework combines convolutional feature extraction, liquid neural networks with learnable time constants, and attention-guided fusion to model temporal EEG dynamics with complementary peripheral physiological and personality features. Dedicated subnetworks are used to process EEG features and auxiliary modalities, and a shared autoencoder-based fusion module is used to learn discriminative latent representations before classification. Subject-dependent experiments conducted on the PhyMER dataset across seven emotional classes achieve an accuracy of 95.45%, surpassing previously reported results. Furthermore, temporal attention analysis provides interpretable insights into emotion-specific temporal relevance, and t-SNE visualizations demonstrate enhanced class separability, highlighting the effectiveness of the proposed approach. Finally, statistical analysis of temporal dynamics confirms that the network self-organizes into distinct functional groups with specialized fast and slow neurons, proving it independently tunes learnable time constants and memory dominance to effectively capture complex emotion artifacts.
Paper Structure (24 sections, 18 equations, 7 figures, 12 tables)

This paper contains 24 sections, 18 equations, 7 figures, 12 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Research Gap and Our Contribution
  4. Methodology
  5. About the Dataset
  6. Proposed Preprocessing Pipeline
  7. EEG Signal Conditioning and Artifact Suppression
  8. Autonomic Nervous System (ANS) Decomposition
  9. Psychometric Contextualization and Data Structuring
  10. Proposed Model
  11. Processing Raw EEG: CNN-LNN-Attention
  12. Feature Encoders for EEG & Peripheral Signals
  13. Autoencoder-based feature fusion
  14. Classification Head
  15. Results
  16. ...and 9 more sections

Figures (7)

  • Figure 1: AF3 channel of EEG data for subject 7, video 24 in each step of preprocessing for the first 5 seconds
  • Figure 2: Detailed Preprocessing Pipeline with Architecture
  • Figure 3: Classification performance analysis: (a) Confusion matrix for test set, (b) Micro and macro-average ROC curves.
  • Figure 4: t-SNE plot before and after training
  • Figure 5: Temporal Attention Analysis of the Liquid Neural Network
  • ...and 2 more figures