READ-Net: Clarifying Emotional Ambiguity via Adaptive Feature Recalibration for Audio-Visual Depression Detection

TL;DR

READ-Net tackles Emotional Ambiguity in audio-visual depression detection by introducing Adaptive Feature Recalibration, a modular framework that disentangles depression-relevant cues from transient emotional noise. It combines Hierarchical Feature Separation, Dual Consistency Regularization, and Asymmetric Distillation to preserve stable depressive signals while filtering context-driven fluctuations, formalizing Emotional Ambiguity with definitions like $\mathcal{A}_\tau$ and $\mathrm{EA}_{\text{err}}$. The method achieves state-of-the-art results on LMVD, D-vlog, and DAIC-WOZ, with average accuracy gains of $4.55\%$ and $1.26\%$ in F1-score, and demonstrates strong robustness to emotional disturbances and compatibility as a plug-in for existing models. The work offers a principled approach to disentangling transient emotions from persistent depressive traits, supporting more reliable and generalizable depression screening in real-world settings.

Abstract

Depression is a severe global mental health issue that impairs daily functioning and overall quality of life. Although recent audio-visual approaches have improved automatic depression detection, methods that ignore emotional cues often fail to capture subtle depressive signals hidden within emotional expressions. Conversely, those incorporating emotions frequently confuse transient emotional expressions with stable depressive symptoms in feature representations, a phenomenon termed \emph{Emotional Ambiguity}, thereby leading to detection errors. To address this critical issue, we propose READ-Net, the first audio-visual depression detection framework explicitly designed to resolve Emotional Ambiguity through Adaptive Feature Recalibration (AFR). The core insight of AFR is to dynamically adjust the weights of emotional features to enhance depression-related signals. Rather than merely overlooking or naively combining emotional information, READ-Net innovatively identifies and preserves depressive-relevant cues within emotional features, while adaptively filtering out irrelevant emotional noise. This recalibration strategy significantly clarifies feature representations, and effectively mitigates the persistent challenge of emotional interference. Additionally, READ-Net can be easily integrated into existing frameworks for improved performance. Extensive evaluations on three publicly available datasets show that READ-Net outperforms state-of-the-art methods, with average gains of 4.55\% in accuracy and 1.26\% in F1-score, demonstrating its robustness to emotional disturbances and improving audio-visual depression detection.

Figures (7)

• Figure 1: Comparison of existing and proposed methods for audio-visual depression detection. (A) Emotion-agnostic methods overlook emotion-embedded depressive cues, leading to incomplete modeling. (B) Directly fused emotional and depressive features risk confusing transient moods with stable depressive signals. (C) Our proposed approach employs Adaptive Feature Recalibration to selectively integrate beneficial emotional cues, enhancing both accuracy and robustness.
• Figure 2: Visualization of Emotional Ambiguity. This figure illustrates the overlap between transient emotional fluctuations (positive and negative) and stable depressive signals over time. The shaded region represents the area of Emotional Ambiguity, where emotional fluctuations complicate the accurate identification of depressive symptoms.
• Figure 3: Overview of READ-Net. It integrates hierarchical feature separation (1), dual consistency regularization (2), and asymmetric distillation (3) to effectively manage Emotional Ambiguity. By dynamically recalibrating emotion-related features and leveraging multimodal data, READ-Net significantly enhances the accuracy and reliability of depression detection, outperforming traditional methods.
• Figure 4: Dual Consistency Regularization (DCR) Workflow. Illustrates DCR’s two stages: the first stage separates $F_d$, $F_n$, and $F_e$ via graph construction, updates, and regularization ($L_{\text{graph}}$); the second stage refines $F_{eDep}$ and $F_{eNonDep}$ using child graph construction, updates, and information flow control ($L_{\text{flow}}$), enhancing discriminability and robustness.
• Figure 5: Progressive Feature Disentanglement after HFS and DCR. The visualization shows how AFR refines depression-related, emotional, and noise features in latent space.