Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhanced Speech Emotion Recognition with Efficient Channel Attention Guided Deep CNN-BiLSTM Framework

Niloy Kumar Kundu, Sarah Kobir, Md. Rayhan Ahmed, Tahmina Aktar, Niloya Roy

TL;DR

This work tackles efficient speech emotion recognition by introducing a dual-channel architecture that pairs attention-guided local feature blocks (ALFBs) with global feature blocks (GFBs). ALFBs leverage 1D CNNs and Efficient Channel Attention to capture salient local cues, while GFBs use BiLSTM layers to model long-range temporal dependencies; the outputs are fused for robust emotion classification. Evaluated on five multilingual datasets (TESS, RAVDESS, BanglaSER, SUBESCO, Emo-DB) with extensive data augmentation, the model achieves state-of-the-art mean accuracies on most benchmarks and demonstrates that local and global feature integration can be achieved with low computational cost. The results suggest significant practical impact for multilingual, resource-constrained SER applications and point toward future multi-modal extensions within constrained budgets.

Abstract

Speech emotion recognition (SER) is crucial for enhancing affective computing and enriching the domain of human-computer interaction. However, the main challenge in SER lies in selecting relevant feature representations from speech signals with lower computational costs. In this paper, we propose a lightweight SER architecture that integrates attention-based local feature blocks (ALFBs) to capture high-level relevant feature vectors from speech signals. We also incorporate a global feature block (GFB) technique to capture sequential, global information and long-term dependencies in speech signals. By aggregating attention-based local and global contextual feature vectors, our model effectively captures the internal correlation between salient features that reflect complex human emotional cues. To evaluate our approach, we extracted four types of spectral features from speech audio samples: mel-frequency cepstral coefficients, mel-spectrogram, root mean square value, and zero-crossing rate. Through a 5-fold cross-validation strategy, we tested the proposed method on five multi-lingual standard benchmark datasets: TESS, RAVDESS, BanglaSER, SUBESCO, and Emo-DB, and obtained a mean accuracy of 99.65%, 94.88%, 98.12%, 97.94%, and 97.19% respectively. The results indicate that our model achieves state-of-the-art (SOTA) performance compared to most existing methods.

Enhanced Speech Emotion Recognition with Efficient Channel Attention Guided Deep CNN-BiLSTM Framework

TL;DR

This work tackles efficient speech emotion recognition by introducing a dual-channel architecture that pairs attention-guided local feature blocks (ALFBs) with global feature blocks (GFBs). ALFBs leverage 1D CNNs and Efficient Channel Attention to capture salient local cues, while GFBs use BiLSTM layers to model long-range temporal dependencies; the outputs are fused for robust emotion classification. Evaluated on five multilingual datasets (TESS, RAVDESS, BanglaSER, SUBESCO, Emo-DB) with extensive data augmentation, the model achieves state-of-the-art mean accuracies on most benchmarks and demonstrates that local and global feature integration can be achieved with low computational cost. The results suggest significant practical impact for multilingual, resource-constrained SER applications and point toward future multi-modal extensions within constrained budgets.

Abstract

Speech emotion recognition (SER) is crucial for enhancing affective computing and enriching the domain of human-computer interaction. However, the main challenge in SER lies in selecting relevant feature representations from speech signals with lower computational costs. In this paper, we propose a lightweight SER architecture that integrates attention-based local feature blocks (ALFBs) to capture high-level relevant feature vectors from speech signals. We also incorporate a global feature block (GFB) technique to capture sequential, global information and long-term dependencies in speech signals. By aggregating attention-based local and global contextual feature vectors, our model effectively captures the internal correlation between salient features that reflect complex human emotional cues. To evaluate our approach, we extracted four types of spectral features from speech audio samples: mel-frequency cepstral coefficients, mel-spectrogram, root mean square value, and zero-crossing rate. Through a 5-fold cross-validation strategy, we tested the proposed method on five multi-lingual standard benchmark datasets: TESS, RAVDESS, BanglaSER, SUBESCO, and Emo-DB, and obtained a mean accuracy of 99.65%, 94.88%, 98.12%, 97.94%, and 97.19% respectively. The results indicate that our model achieves state-of-the-art (SOTA) performance compared to most existing methods.

Paper Structure

This paper contains 19 sections, 21 equations, 10 figures, 11 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Methodology
  4. Proposed Dual Channel SER Model
  5. Attention Based Local Feature Blocks
  6. Global Feature Blocks
  7. Dataset and Feature Extraction
  8. Dataset
  9. Data Augmentation
  10. Extracted Features
  11. Experimental Analysis
  12. Experimental Setup
  13. Hyperparameter Tuning
  14. Result Analysis
  15. Evaluation Metrics
  16. ...and 4 more sections

Figures (10)

  • Figure 1: A graphical illustration of the proposed SER model's generic workflow.
  • Figure 2: Overview of the proposed approach for SER.
  • Figure 3: Visual representation of the proposed dual-channel architecture.
  • Figure 4: Visual representation of the Squeeze and Excitation Network block hu2018squeeze.
  • Figure 5: Visual representation of the Efficient Channel Attention Network block wang2020eca.
  • ...and 5 more figures