Enhancing Speech Emotion Recognition using Dynamic Spectral Features and Kalman Smoothing

TL;DR

Speech emotion recognition often suffers from frame-level jitter due to noise and temporal variability. The authors introduce KF-TSER, a hybrid framework that combines Dynamic Spectral Features ($\Delta$ and $\Delta\Delta$) with Kalman smoothing to enforce temporal continuity and stabilize predictions. On the RAVDESS dataset with four emotions, KF-TSER achieves an accuracy of 87% and outperforms baseline frame-level classifiers, notably improving distinctions between similar arousal states such as Happy and Angry. The approach uses a lightweight 41-dimensional feature vector and provides publicly available code, highlighting practical, real-time applicability and improved robustness against noise.

Abstract

Speech Emotion Recognition systems often use static features like Mel-Frequency Cepstral Coefficients (MFCCs), Zero Crossing Rate (ZCR), and Root Mean Square Energy (RMSE). Because of this, they can misclassify emotions when there is acoustic noise in vocal signals. To address this, we added dynamic features using Dynamic Spectral features (Deltas and Delta-Deltas) along with the Kalman Smoothing algorithm. This approach reduces noise and improves emotion classification. Since emotion changes over time, the Kalman Smoothing filter also helped make the classifier outputs more stable. Tests on the RAVDESS dataset showed that this method achieved a state-of-the-art accuracy of 87\% and reduced misclassification between emotions with similar acoustic features

Figures (4)

• Figure 1: KF-TSER Pipeline
• Figure 2: Temporal stabilization of emotion probabilities. case of "Happy"
• Figure 3: Confusion Matrix of the proposed system achieving 87% accuracy.
• Figure 4: Training Convergence and Performance Gain Analysis