Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Non-invasive Neural Decoding in Source Reconstructed Brain Space

Yonatan Gideoni, Ryan Charles Timms, Oiwi Parker Jones

TL;DR

It is shown that using established techniques to reconstruct the sensors' sources' neural activity it is possible to decode from voxels for MEG data as well, which enables spatial inductive biases, spatial data augmentations, better interpretability, zero-shot generalisation between datasets, and data harmonisation.

Abstract

Non-invasive brainwave decoding is usually done using Magneto/Electroencephalography (MEG/EEG) sensor measurements as inputs. This makes combining datasets and building models with inductive biases difficult as most datasets use different scanners and the sensor arrays have a nonintuitive spatial structure. In contrast, fMRI scans are acquired directly in brain space, a voxel grid with a typical structured input representation. By using established techniques to reconstruct the sensors' sources' neural activity it is possible to decode from voxels for MEG data as well. We show that this enables spatial inductive biases, spatial data augmentations, better interpretability, zero-shot generalisation between datasets, and data harmonisation.

Non-invasive Neural Decoding in Source Reconstructed Brain Space

TL;DR

It is shown that using established techniques to reconstruct the sensors' sources' neural activity it is possible to decode from voxels for MEG data as well, which enables spatial inductive biases, spatial data augmentations, better interpretability, zero-shot generalisation between datasets, and data harmonisation.

Abstract

Non-invasive brainwave decoding is usually done using Magneto/Electroencephalography (MEG/EEG) sensor measurements as inputs. This makes combining datasets and building models with inductive biases difficult as most datasets use different scanners and the sensor arrays have a nonintuitive spatial structure. In contrast, fMRI scans are acquired directly in brain space, a voxel grid with a typical structured input representation. By using established techniques to reconstruct the sensors' sources' neural activity it is possible to decode from voxels for MEG data as well. We show that this enables spatial inductive biases, spatial data augmentations, better interpretability, zero-shot generalisation between datasets, and data harmonisation.

Paper Structure

This paper contains 18 sections, 5 figures, 14 tables.

Table of Contents

  1. Introduction
  2. Preprocessing and Source Reconstruction
  3. Source vs Sensor Space
  4. Using Source Space for Inductive Bias
  5. Spatial Data Augmentations
  6. Impact of Different Brain Regions
  7. Zero-shot Interdataset Generalisation
  8. Learning from Combined Datasets
  9. Discussion and Future Work
  10. Preprocessing Visual Sanity
  11. Preprocessing Ablations
  12. Hyperparameters
  13. Region Masking
  14. Region Functions
  15. Subjects 002 and 003
  16. ...and 3 more sections

Figures (5)

  • Figure 1: General setup --- MEG data is measured using sensors, preprocessed, and used for decoding. Here we use source reconstruction to learn over estimated neural activity instead.
  • Figure 2: Illustration of slice dropout (top) and cube masking (bottom). Shaded voxels are set to zero.
  • Figure 3: Accuracy when masking out different brain regions, shown here for the model trained on subject 001 in the Armeni dataset. Other subjects are shown in Appendix \ref{['app:regmask']}. "Baseline" corresponds to no masking. For all single-subject models the baseline performs better than most masks. Errors are standard deviations over models trained on different seeds.
  • Figure 4: Epoched sensor activity for the final preprocessing pipeline. The topology activity maps are qualitatively similar to those found by capilla2013early. Averaged data for subject 1 session 1 in the Armeni dataset is shown.
  • Figure 5: Accuracy when masking out different brain regions for (a) subject 002 and (b) subject 003.