Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MEG-to-MEG Transfer Learning and Cross-Task Speech/Silence Detection with Limited Data

Xabier de Zuazo, Vincenzo Verbeni, Eva Navas, Ibon Saratxaga, Mathieu Bourguignon, Nicola Molinaro

TL;DR

This work tackles data-efficiency in MEG-based speech decoding by pre-training a Conformer-style model on 50 hours of single-subject listening MEG data and fine-tuning it on only about 5 minutes per subject across 18 participants for perception and production tasks. The key finding is that large-scale pre-training improves both in-task performance and cross-task generalization, enabling cross-task decoding between listening, playback, and production, with cross-task gains up to 5–6%. Importantly, production-trained models can decode passive listening above chance, indicating shared neural representations beyond task-specific motor activity. The study advances practical MEG-based neurotechnologies by demonstrating data-efficient transfer learning and highlighting asymmetries in cross-task transfer that reflect motor planning involvement in production.

Abstract

Data-efficient neural decoding is a central challenge for speech brain-computer interfaces. We present the first demonstration of transfer learning and cross-task decoding for MEG-based speech models spanning perception and production. We pre-train a Conformer-based model on 50 hours of single-subject listening data and fine-tune on just 5 minutes per subject across 18 participants. Transfer learning yields consistent improvements, with in-task accuracy gains of 1-4% and larger cross-task gains of up to 5-6%. Not only does pre-training improve performance within each task, but it also enables reliable cross-task decoding between perception and production. Critically, models trained on speech production decode passive listening above chance, confirming that learned representations reflect shared neural processes rather than task-specific motor activity.

MEG-to-MEG Transfer Learning and Cross-Task Speech/Silence Detection with Limited Data

TL;DR

This work tackles data-efficiency in MEG-based speech decoding by pre-training a Conformer-style model on 50 hours of single-subject listening MEG data and fine-tuning it on only about 5 minutes per subject across 18 participants for perception and production tasks. The key finding is that large-scale pre-training improves both in-task performance and cross-task generalization, enabling cross-task decoding between listening, playback, and production, with cross-task gains up to 5–6%. Importantly, production-trained models can decode passive listening above chance, indicating shared neural representations beyond task-specific motor activity. The study advances practical MEG-based neurotechnologies by demonstrating data-efficient transfer learning and highlighting asymmetries in cross-task transfer that reflect motor planning involvement in production.

Abstract

Data-efficient neural decoding is a central challenge for speech brain-computer interfaces. We present the first demonstration of transfer learning and cross-task decoding for MEG-based speech models spanning perception and production. We pre-train a Conformer-based model on 50 hours of single-subject listening data and fine-tune on just 5 minutes per subject across 18 participants. Transfer learning yields consistent improvements, with in-task accuracy gains of 1-4% and larger cross-task gains of up to 5-6%. Not only does pre-training improve performance within each task, but it also enables reliable cross-task decoding between perception and production. Critically, models trained on speech production decode passive listening above chance, confirming that learned representations reflect shared neural processes rather than task-specific motor activity.
Paper Structure (11 sections, 3 figures, 3 tables)

This paper contains 11 sections, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methods
  4. Datasets
  5. Model Architecture
  6. Experimental Protocol
  7. Results
  8. In-Task Transfer Learning
  9. Cross-Task Decoding Baseline
  10. Cross-Task with Transfer Learning
  11. Discussion and Conclusion

Figures (3)

  • Figure 1: Effect size of transfer learning for cross-task decoding. Bars show the mean improvement in F1 score (transfer learning minus training from scratch) for each task and train-test task pair. Error bars indicate the standard deviation across subjects.
  • Figure 2: Cross-task F1 asymmetry for each task pair. Each x-axis category shows two directions: circle markers correspond to the left-to-right direction in the label, and diamond markers to the reverse direction. For each model, the two direction-specific points are connected to highlight asymmetry. Error bars indicate the standard deviation across subjects.
  • Figure 3: Subject-level in-task F1 improvement with transfer learning. Each bar shows the improvement in F1 score (transfer learning minus training from scratch) for a single subject and task. Colors indicate the three tasks.