A Convolutional Framework for Mapping Imagined Auditory MEG into Listened Brain Responses

TL;DR

This work addresses decoding imagined auditory content from MEG by linking imagined responses to the listened-response space, enabling models trained on perception to decode imagination. It first demonstrates within-subject feasibility using a sliding-window linear model, then overcoming cross-subject variability with an encoder–decoder CNN that includes a subject-specific calibration layer for generalization to unseen participants. The results show the CNN outperforms null models in predicting listened MEG from imagined MEG, establishing a foundation for noninvasive decoding of imagined speech and music. The approach holds promise for future brain–computer interfaces that translate imagined auditory content into perceivable neural representations.

Abstract

Decoding imagined speech engages complex neural processes that are difficult to interpret due to uncertainty in timing and the limited availability of imagined-response datasets. In this study, we present a Magnetoencephalography (MEG) dataset collected from trained musicians as they imagined and listened to musical and poetic stimuli. We show that both imagined and perceived brain responses contain consistent, condition-specific information. Using a sliding-window ridge regression model, we first mapped imagined responses to listened responses at the single-subject level, but found limited generalization across subjects. At the group level, we developed an encoder-decoder convolutional neural network with a subject-specific calibration layer that produced stable and generalizable mappings. The CNN consistently outperformed the null model, yielding significantly higher correlations between predicted and true listened responses for nearly all held-out subjects. Our findings demonstrate that imagined neural activity can be transformed into perception-like responses, providing a foundation for future brain-computer interface applications involving imagined speech and music.

Figures (7)

• Figure 1: MEG Experiment Paradigm. Some trials were listening condition (top) and the others were imagery condition (bottom). Participants listened to and imagined two melodies and two poem snippets.
• Figure 2: A) Trial-by-trial representational similarity matrix computed across all MEG trials. Each 10×10 block corresponds to correlations within and between the eight experimental conditions. B) Block-averaged representational similarity matrix (8×8), showing the mean correlation between conditions. C) Confusion matrix for the correlation-based classifier. Each cell shows the percentage of trials assigned to each predicted condition, row-normalized so that values represent correctly classified (%) for each true condition.
• Figure 3: Linear mapping performance across subjects. Mean sliding-window correlations between the predicted listened response and the ground-truth listened signal (Real) are shown alongside correlations from the shuffled null model (Null). Error bars indicate the standard error computed across windows and channels.
• Figure 4: Linear mapping. Histogram of correlation coefficients between the predicted listened response and all listened trials. Blue: correlations with trials from the same class. Red: correlations with trials from all other classes.
• Figure 5: Schematic of the encoder–decoder CNN and subject-specific calibration module used to map imagined to listened MEG responses. The encoder compresses the 155-channel input features, and the decoder reconstructs a 155-channel estimate of the listened response. A calibration layer adapts the shared model to each individual subject before final prediction.