Efficient Long Speech Sequence Modelling for Time-Domain Depression Level Estimation

TL;DR

This work tackles the problem of estimating depression severity from speech by addressing the limitations of time-frequency representations and short segments. It introduces a time-domain framework that processes long speech sequences using a dual-path Bi-Mamba–SSM long-sequence module, enhanced by a temporal external attention mechanism, and paired with a prediction module to output depression scores. The approach yields state-of-the-art RMSE and MAE on AVEC2013 and AVEC2014, especially as input length increases, demonstrating improved robustness to real-world long utterances. The method's ability to reconstruct and emphasize depression-relevant cues in raw waveforms has practical implications for scalable, real-time mental health screening from naturalistic speech data.

Abstract

Depression significantly affects emotions, thoughts, and daily activities. Recent research indicates that speech signals contain vital cues about depression, sparking interest in audio-based deep-learning methods for estimating its severity. However, most methods rely on time-frequency representations of speech which have recently been criticized for their limitations due to the loss of information when performing time-frequency projections, e.g. Fourier transform, and Mel-scale transformation. Furthermore, segmenting real-world speech into brief intervals risks losing critical interconnections between recordings. Additionally, such an approach may not adequately reflect real-world scenarios, as individuals with depression often pause and slow down in their conversations and interactions. Building on these observations, we present an efficient method for depression level estimation using long speech signals in the time domain. The proposed method leverages a state space model coupled with the dual-path structure-based long sequence modelling module and temporal external attention module to reconstruct and enhance the detection of depression-related cues hidden in the raw audio waveforms. Experimental results on the AVEC2013 and AVEC2014 datasets show promising results in capturing consequential long-sequence depression cues and demonstrate outstanding performance over the state-of-the-art.

Figures (3)

• Figure 1: (a) Overall framework of the proposed methods which consists of the long sequence modelling module, the temporal external attention module, and the prediction module. (b) Illustration of the long sequence modelling module. It employs the Bi-Mamba and dual-path architecture to reconstruct the raw audio waves. (c) Illustration of the temporal external attention module. It enhances depression patterns through correlation analysis with additional parameters. (d) Illustration of the prediction module. It predicts the individual depression level scores based on previous output.
• Figure 2: The Bi-Mamba network structure. $\textbf{g}^{\mathbf{+}}$ and $\textbf{j}^{\mathbf{+}}$ stand for the anterior processed sequence. $\textbf{g}^{\mathbf{-}}$ and $\textbf{j}^{\mathbf{-}}$ stand for the posterior processed sequence.
• Figure 3: Comparison of network performance based on RMSE and MAE across different speech lengths in the AVEC14 dataset.