Fine-tuning Pre-trained Audio Models for COVID-19 Detection: A Technical Report

TL;DR

This study evaluates fine-tuned pre-trained audio models (Audio-MAE and PANNs CNN6/10/14) on two COVID-19 benchmarks, Coswara and Coughvid. It introduces strict demographic stratification via age and gender to prevent leakage and to obtain realistic generalization estimates. Intra-dataset results show moderate performance, with Audio-MAE excelling on Coswara, but cross-dataset evaluations reveal severe generalization failures, highlighting dataset size and demographic confounds as major barriers. The findings emphasize the need for rigorous demographic controls and larger, more representative datasets to develop truly generalizable audio-based COVID-19 detectors.

Abstract

This technical report investigates the performance of pre-trained audio models on COVID-19 detection tasks using established benchmark datasets. We fine-tuned Audio-MAE and three PANN architectures (CNN6, CNN10, CNN14) on the Coswara and COUGHVID datasets, evaluating both intra-dataset and cross-dataset generalization. We implemented a strict demographic stratification by age and gender to prevent models from exploiting spurious correlations between demographic characteristics and COVID-19 status. Intra-dataset results showed moderate performance, with Audio-MAE achieving the strongest result on Coswara (0.82 AUC, 0.76 F1-score), while all models demonstrated limited performance on Coughvid (AUC 0.58-0.63). Cross-dataset evaluation revealed severe generalization failure across all models (AUC 0.43-0.68), with Audio-MAE showing strong performance degradation (F1-score 0.00-0.08). Our experiments demonstrate that demographic balancing, while reducing apparent model performance, provides more realistic assessment of COVID-19 detection capabilities by eliminating demographic leakage - a confounding factor that inflate performance metrics. Additionally, the limited dataset sizes after balancing (1,219-2,160 samples) proved insufficient for deep learning models that typically require substantially larger training sets. These findings highlight fundamental challenges in developing generalizable audio-based COVID-19 detection systems and underscore the importance of rigorous demographic controls for clinically robust model evaluation.