CAF-Mamba: Mamba-Based Cross-Modal Adaptive Attention Fusion for Multimodal Depression Detection
Bowen Zhou, Marc-André Fiedler, Ayoub Al-Hamadi
TL;DR
CAF-Mamba tackles multimodal depression detection by explicitly modeling cross-modal interactions and adaptively fusing modalities via a Mamba-based architecture. It introduces a Unimodal Extraction Module, a Cross-modal Interaction Mamba Encoder, and an Adaptive Attention Mamba Fusion Module to capture both inter-modal dependencies and long-range temporal dynamics. Empirical results on in-the-wild LMVD and D-Vlog show state-of-the-art performance and favorable efficiency, with ablations validating the contributions of explicit cross-modal interaction and modality-wise fusion. The framework is extensible to additional modalities, promising robust, real-time applicability in real-world depression detection systems.
Abstract
Depression is a prevalent mental health disorder that severely impairs daily functioning and quality of life. While recent deep learning approaches for depression detection have shown promise, most rely on limited feature types, overlook explicit cross-modal interactions, and employ simple concatenation or static weighting for fusion. To overcome these limitations, we propose CAF-Mamba, a novel Mamba-based cross-modal adaptive attention fusion framework. CAF-Mamba not only captures cross-modal interactions explicitly and implicitly, but also dynamically adjusts modality contributions through a modality-wise attention mechanism, enabling more effective multimodal fusion. Experiments on two in-the-wild benchmark datasets, LMVD and D-Vlog, demonstrate that CAF-Mamba consistently outperforms existing methods and achieves state-of-the-art performance.