EEG-based Cognitive Load Estimation of Acoustic Parameters for Data Sonification

TL;DR

This work targets the problem of measuring cognitive load (CL) elicited by psychoacoustic parameters used in data sonification of astronomical imagery. It combines multiple EEG descriptors (PSD, spectral topography, and STFT) with a suite of learning architectures, including temporal CNNs, spatial CNNs, and a spatio-temporal fusion model, to predict high/low CL from EEG data. A convolutional Siamese network is used to assess perceptual similarity between parameter-induced EEG embeddings, revealing that parameters causing similar CL tend to yield nearby latent representations, and that extremum focus-levels are more readily detected than intermediate ones. The results demonstrate that EEG-based CL estimation is feasible and accurate (peak $F1$-score near 0.98 with fusion) and that the approach can help benchmark and refine sonification mappings for data exploration and accessibility, while acknowledging limitations such as a limited parameter set and subject-specific variability.

Abstract

Sonification is a data visualization technique which expresses data attributes via psychoacoustic parameters, which are non-speech audio signals used to convey information. This paper investigates the binary estimation of cognitive load induced by psychoacoustic parameters conveying the focus level of an astronomical image via Electroencephalogram (EEG) embeddings. Employing machine learning and deep learning methodologies, we demonstrate that EEG signals are reliable for (a) binary estimation of cognitive load, (b) isolating easy vs difficult visual-to-auditory perceptual mappings, and (c) capturing perceptual similarities among psychoacoustic parameters. Our key findings reveal that (1) EEG embeddings can reliably measure cognitive load, achieving a peak F1-score of 0.98; (2) Extreme focus levels are easier to detect via auditory mappings than intermediate ones, and (3) psychoacoustic parameters inducing comparable cognitive load levels tend to generate similar EEG encodings.

Figures (8)

• Figure 1: Images of the Messier 110 galaxy corresponding to focus levels ranging from 1--10 are presented from left to right.
• Figure 2: Overview of the Immediate Recall (IR) and Compared Recall (CR) protocols. Refer to section \ref{['E_Proto']} for details.
• Figure 3: Power spectral density vectors and spectral topographic plots. Refer to sections \ref{['PSDV']} and \ref{['STP']} for details.
• Figure 4: Temporal-CNN Architecture. Refer to section \ref{['T_CNN']} for details.
• Figure 5: Spatial CNN Architectures. Refer to section \ref{['S_CNN']} for details.