Optimizing Domain-Adaptive Self-Supervised Learning for Clinical Voice-Based Disease Classification

TL;DR

Data scarcity and domain mismatch hinder deep learning for clinical voice analysis. The authors propose domain-adaptive self-supervised learning by pre-training an Audio Spectrogram Transformer (AST) with Masked Autoencoders (MAE) on in-domain pathological voices, and they systematically optimize three MAE components: reconstruction loss, input normalization, and masking strategy. Their optimized configuration—MA-Error loss, patch-wise normalization, and content-aware masking—achieves Macro F1 of $0.688 \pm 0.009$ on Bridge2AI-Voice after fine-tuning, outperforming an out-of-domain SSL baseline (Macro F1 $0.663 \pm 0.011$). This demonstrates that objective-level tailoring to data characteristics can surpass larger general-domain pre-training in data-constrained medical audio tasks, with implications for robust, multimodal disease screening from voice.

Abstract

The human voice is a promising non-invasive digital biomarker, yet deep learning for voice-based health analysis is hindered by data scarcity and domain mismatch, where models pre-trained on general audio fail to capture the subtle pathological features characteristic of clinical voice data. To address these challenges, we investigate domain-adaptive self-supervised learning (SSL) with Masked Autoencoders (MAE) and demonstrate that standard configurations are suboptimal for health-related audio. Using the Bridge2AI-Voice dataset, a multi-institutional collection of pathological voices, we systematically examine three performance-critical factors: reconstruction loss (Mean Absolute Error vs. Mean Squared Error), normalization (patch-wise vs. global), and masking (random vs. content-aware). Our optimized design, which combines Mean Absolute Error (MA-Error) loss, patch-wise normalization, and content-aware masking, achieves a Macro F1 of $0.688 \pm 0.009$ (over 10 fine-tuning runs), outperforming a strong out-of-domain SSL baseline pre-trained on large-scale general audio, which has a Macro F1 of $0.663 \pm 0.011$. The results show that MA-Error loss improves robustness and content-aware masking boosts performance by emphasizing information-rich regions. These findings highlight the importance of component-level optimization in data-constrained medical applications that rely on audio data.

Figures (1)

• Figure 1: An overview of the two-stage framework. Stage 1 (Domain-Adaptive SSL Pre-training): An AST Encoder is pre-trained by reconstructing masked spectrogram patches, with its weights optimized via a reconstruction loss. Stage 2 (Downstream Multi-Label Classification): The learned encoder serves as a feature extractor, producing deep features that are fused with static acoustic features for the final prediction using an Attention-FFNN.