SMSAT: A Multimodal Acoustic Dataset and Deep Contrastive Learning Framework for Affective and Physiological Modeling of Spiritual Meditation

TL;DR

This work tackles the need for objective evaluation of auditory stimuli on affective and physiological states by introducing SMSAT, a diverse acoustic time-series dataset covering spiritual meditation, music, and silence. It combines a self-supervised SMSAT ATS encoder with contrastive learning and a deep Calmness Analysis Model (CAM) to achieve near-perfect discrimination among auditory conditions. Rigorous signal validation, statistical analyses, and visualization substantiate that spiritual meditation closely resembles resting states, while music elicits greater physiological fluctuations. The dataset and the scalable modeling framework offer practical impact for stress monitoring, mental well-being, and therapy via audio-based interventions, outperforming prior state-of-the-art methods by a substantial margin.

Abstract

Understanding how auditory stimuli influence emotional and physiological states is fundamental to advancing affective computing and mental health technologies. In this paper, we present a multimodal evaluation of the affective and physiological impacts of three auditory conditions, that is, spiritual meditation (SM), music (M), and natural silence (NS), using a comprehensive suite of biometric signal measures. To facilitate this analysis, we introduce the Spiritual, Music, Silence Acoustic Time Series (SMSAT) dataset, a novel benchmark comprising acoustic time series (ATS) signals recorded under controlled exposure protocols, with careful attention to demographic diversity and experimental consistency. To model the auditory induced states, we develop a contrastive learning based SMSAT audio encoder that extracts highly discriminative embeddings from ATS data, achieving 99.99% classification accuracy in interclass and intraclass evaluations. Furthermore, we propose the Calmness Analysis Model (CAM), a deep learning framework integrating 25 handcrafted and learned features for affective state classification across auditory conditions, attaining robust 99.99% classification accuracy. In contrast, pairwise t tests reveal significant deviations in cardiac response characteristics (CRC) between SM analysis via ANOVA inducing more significant physiological fluctuations. Compared to existing state of the art methods reporting accuracies up to 90%, the proposed model demonstrates substantial performance gains (up to 99%). This work contributes a validated multimodal dataset and a scalable deep learning framework for affective computing applications in stress monitoring, mental well-being, and therapeutic audio-based interventions.

Figures (8)

• Figure 1: Detailed flow graph of the proposed methodology.
• Figure 2: (a) Data setup (b) SMSAT Statistics (c) Time domain plot.
• Figure 3: Time domain and FFT comparison of raw and theoretical ATS signals.
• Figure 4: Data augmentation techniques applied to ATS signals.
• Figure 5: SMSAT ATS encoder (a) Architecture (b) Visualization of layers for ATS signals.