Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Domain Adaptation-Enhanced Searchlight: Enabling classification of brain states from visual perception to mental imagery

Alexander Olza, David Soto, Roberto Santana

TL;DR

This work tackles cross-domain brain decoding, aiming to transfer classifiers trained on visual perception to mental imagery in fMRI data. It formalizes Domain Adaptation (DA) and evaluates 15 DA methods within a perception-to-imagery cross-domain framework, incorporating a DA-enhanced searchlight to map distributed neural representations. Across 18 subjects and 14 ROIs, DA significantly improves imagery decoding in both binary and multiclass settings, with RTLC often providing the strongest gains and DA-enhanced searchlight revealing widespread involvement of visual and frontoparietal regions. The study provides robust methodological validation via permutation testing, demonstrates the practical value of DA for BCIs and neurofeedback, and shares code and data publicly to foster further research.

Abstract

In cognitive neuroscience and brain-computer interface research, accurately predicting imagined stimuli is crucial. This study investigates the effectiveness of Domain Adaptation (DA) in enhancing imagery prediction using primarily visual data from fMRI scans of 18 subjects. Initially, we train a baseline model on visual stimuli to predict imagined stimuli, utilizing data from 14 brain regions. We then develop several models to improve imagery prediction, comparing different DA methods. Our results demonstrate that DA significantly enhances imagery prediction in binary classification on our dataset, as well as in multiclass classification on a publicly available dataset. We then conduct a DA-enhanced searchlight analysis, followed by permutation-based statistical tests to identify brain regions where imagery decoding is consistently above chance across subjects. Our DA-enhanced searchlight predicts imagery contents in a highly distributed set of brain regions, including the visual cortex and the frontoparietal cortex, thereby outperforming standard cross-domain classification methods. The complete code and data for this paper have been made openly available for the use of the scientific community.

Domain Adaptation-Enhanced Searchlight: Enabling classification of brain states from visual perception to mental imagery

TL;DR

This work tackles cross-domain brain decoding, aiming to transfer classifiers trained on visual perception to mental imagery in fMRI data. It formalizes Domain Adaptation (DA) and evaluates 15 DA methods within a perception-to-imagery cross-domain framework, incorporating a DA-enhanced searchlight to map distributed neural representations. Across 18 subjects and 14 ROIs, DA significantly improves imagery decoding in both binary and multiclass settings, with RTLC often providing the strongest gains and DA-enhanced searchlight revealing widespread involvement of visual and frontoparietal regions. The study provides robust methodological validation via permutation testing, demonstrates the practical value of DA for BCIs and neurofeedback, and shares code and data publicly to foster further research.

Abstract

In cognitive neuroscience and brain-computer interface research, accurately predicting imagined stimuli is crucial. This study investigates the effectiveness of Domain Adaptation (DA) in enhancing imagery prediction using primarily visual data from fMRI scans of 18 subjects. Initially, we train a baseline model on visual stimuli to predict imagined stimuli, utilizing data from 14 brain regions. We then develop several models to improve imagery prediction, comparing different DA methods. Our results demonstrate that DA significantly enhances imagery prediction in binary classification on our dataset, as well as in multiclass classification on a publicly available dataset. We then conduct a DA-enhanced searchlight analysis, followed by permutation-based statistical tests to identify brain regions where imagery decoding is consistently above chance across subjects. Our DA-enhanced searchlight predicts imagery contents in a highly distributed set of brain regions, including the visual cortex and the frontoparietal cortex, thereby outperforming standard cross-domain classification methods. The complete code and data for this paper have been made openly available for the use of the scientific community.
Paper Structure (25 sections, 1 equation, 19 figures, 4 tables)

This paper contains 25 sections, 1 equation, 19 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Methodological background
  3. Related Work
  4. ML techniques for investigating commonalities in brain mechanisms for visual and imagery tasks
  5. Other applications of DA in neuroscience
  6. Description of the cross-domain analysis
  7. Experimental setup
  8. Machine learning task and validation framework
  9. Evaluation metrics and statistical validation
  10. Initial comparison of DA techniques
  11. Evaluation of the searchlight analysis
  12. Experiments
  13. Comparison of DA techniques
  14. Analysis of individual ROIs using RTLC
  15. DA-enhanced searchlight analysis
  16. ...and 10 more sections

Figures (19)

  • Figure 1: Diagram of the experimental pipeline. Data acquisition: In the perception phase, fMRI data from 18 healthy subjects was acquired while they were presented with pictures of Living (Dog) and Non-living (Scissor) items. In the imagery phase, fMRI scans were recorded while the participants vividly imagined items from each category. DA comparison in the ROIs: The voxels from 14 anatomical Regions of Interest (ROIs) were used to compare 15 DA methods against a Logistic Regression (LR) baseline trained solely on perception and an alternative LR trained on perception and imagery. All the methods were tested on imagery. The comparison was repeated on a publicly available multiclass dataset. DA-enhanced searchlight: A local DA classifier was trained on the vicinity of each voxel in the brain, using data from perception and imagery, to produce a three-dimensional map of classifier accuracy. Created in BioRender. O, A. (2025) https://BioRender.com/u30c785
  • Figure 2: Critical Difference diagram. The scale above shows the average ranking of each algorithm across all observations, and the horizontal lines group together algorithms that are not statistically different.
  • Figure 3: Frequency table. For each cell $(i,j)$, the numerical value shows for how many subjects algorithm $i$ was significantly superior to algorithm $j$. The sum of each row accounts for the number of times each algorithm was significantly better than others, and the sum of each column shows how many times each algorithm was significantly worse than others.
  • Figure 4: CD diagram of RTLC performance in the imagery domain across individual ROIs. FFG: Fusiform Gyrus; LOG: Lateral occipital gyrus; ITG: Inferior temporal gyrus; TP: Temporal pole; PCG: Posterior cyngulate gyrus; PCUN: Precuneus; IPL: Inferior parietal lobe; MTG: Middle temporal gyrus; SFG: Superior frontal gyrus; IFGoperc: Inferior frontal gyrus, pars opercularis, IFGtriang: Inferior frontal gyrus, pars triangularis; IFGorbital: Inferior frontal gyrus, pars orbitalis; FP: Frontopolar cortex; MOG: Medial orbital gyrus.
  • Figure 5: CD diagram of baseline performance in the perception domain across individual ROIs. FFG: Fusiform Gyrus; LOG: Lateral occipital gyrus; ITG: Inferior temporal gyrus; TP: Temporal pole; PCG: Posterior cyngulate gyrus; PCUN: Precuneus; IPL: Inferior parietal lobe; MTG: Middle temporal gyrus; SFG: Superior frontal gyrus; IFGoperc: Inferior frontal gyrus, pars opercularis, IFGtriang: Inferior frontal gyrus, pars triangularis; IFGorbital: Inferior frontal gyrus, pars orbitalis; FP: Frontopolar cortex; MOG: Medial orbital gyrus.
  • ...and 14 more figures