Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Differentially Private Multimodal Laplacian Dropout (DP-MLD) for EEG Representative Learning

Xiaowen Fu, Bingxin Wang, Xinzhou Guo, Guoqing Liu, Yang Xiang

TL;DR

This work addresses privacy concerns in multimodal EEG learning for FoG detection by introducing DP-MLD, a framework that treats EEG as text with Bert and other modalities as images with ViT, fused via cross-attention. It introduces an adaptive, per-feature Laplacian dropout DP scheme and a two-step optimization driven by Gumbel-Softmax to allocate the privacy budget, achieving strong privacy guarantees while maintaining high predictive accuracy. Empirically, DP-MLD attains about a 4% accuracy boost under DP on a FoG PD dataset, and a non-private variant approaches near 100% accuracy, demonstrating both privacy protection and practical effectiveness. The approach offers a scalable blueprint for privacy-preserving multimodal EEG learning with cross-modal fusion, with potential applicability to other clinical multimodal datasets and privacy frameworks.

Abstract

Recently, multimodal electroencephalogram (EEG) learning has shown great promise in disease detection. At the same time, ensuring privacy in clinical studies has become increasingly crucial due to legal and ethical concerns. One widely adopted scheme for privacy protection is differential privacy (DP) because of its clear interpretation and ease of implementation. Although numerous methods have been proposed under DP, it has not been extensively studied for multimodal EEG data due to the complexities of models and signal data considered there. In this paper, we propose a novel Differentially Private Multimodal Laplacian Dropout (DP-MLD) scheme for multimodal EEG learning. Our approach proposes a novel multimodal representative learning model that processes EEG data by language models as text and other modal data by vision transformers as images, incorporating well-designed cross-attention mechanisms to effectively extract and integrate cross-modal features. To achieve DP, we design a novel adaptive feature-level Laplacian dropout scheme, where randomness allocation and performance are dynamically optimized within given privacy budgets. In the experiment on an open-source multimodal dataset of Freezing of Gait (FoG) in Parkinson's Disease (PD), our proposed method demonstrates an approximate 4\% improvement in classification accuracy, and achieves state-of-the-art performance in multimodal EEG learning under DP.

Differentially Private Multimodal Laplacian Dropout (DP-MLD) for EEG Representative Learning

TL;DR

This work addresses privacy concerns in multimodal EEG learning for FoG detection by introducing DP-MLD, a framework that treats EEG as text with Bert and other modalities as images with ViT, fused via cross-attention. It introduces an adaptive, per-feature Laplacian dropout DP scheme and a two-step optimization driven by Gumbel-Softmax to allocate the privacy budget, achieving strong privacy guarantees while maintaining high predictive accuracy. Empirically, DP-MLD attains about a 4% accuracy boost under DP on a FoG PD dataset, and a non-private variant approaches near 100% accuracy, demonstrating both privacy protection and practical effectiveness. The approach offers a scalable blueprint for privacy-preserving multimodal EEG learning with cross-modal fusion, with potential applicability to other clinical multimodal datasets and privacy frameworks.

Abstract

Recently, multimodal electroencephalogram (EEG) learning has shown great promise in disease detection. At the same time, ensuring privacy in clinical studies has become increasingly crucial due to legal and ethical concerns. One widely adopted scheme for privacy protection is differential privacy (DP) because of its clear interpretation and ease of implementation. Although numerous methods have been proposed under DP, it has not been extensively studied for multimodal EEG data due to the complexities of models and signal data considered there. In this paper, we propose a novel Differentially Private Multimodal Laplacian Dropout (DP-MLD) scheme for multimodal EEG learning. Our approach proposes a novel multimodal representative learning model that processes EEG data by language models as text and other modal data by vision transformers as images, incorporating well-designed cross-attention mechanisms to effectively extract and integrate cross-modal features. To achieve DP, we design a novel adaptive feature-level Laplacian dropout scheme, where randomness allocation and performance are dynamically optimized within given privacy budgets. In the experiment on an open-source multimodal dataset of Freezing of Gait (FoG) in Parkinson's Disease (PD), our proposed method demonstrates an approximate 4\% improvement in classification accuracy, and achieves state-of-the-art performance in multimodal EEG learning under DP.
Paper Structure (22 sections, 1 theorem, 19 equations, 5 figures, 1 table, 2 algorithms)

This paper contains 22 sections, 1 theorem, 19 equations, 5 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. RELATED WORKS
  3. Multimodal EEG
  4. Differential Privacy
  5. Multimodal Deep Learning under DP
  6. Gumbel-Softmax
  7. DP-MLD: Multimodal Laplacian Dropout with Differential Privacy
  8. Multimodal Representative Learning
  9. EEG Feature Learning via Bert
  10. Other Modal (OM) Feature Learning via ViT
  11. Cross-Modal Feature Learning
  12. Laplacian Dropout DP Scheme
  13. Element-Wise Laplacian Dropout
  14. Two-step Optimization for training
  15. Algorithm Summary
  16. ...and 7 more sections

Key Result

Theorem 1

Given the feature $\mathbf{f}$, suppose $\mathcal{M}(\mathbf{f}) = \mathbf{f} + \mathbf{r}$ is $\varepsilon^{'}$-DP by adding element wise Laplacian noise $\mathbf{r} = (r_1,\cdots,r_k)$ with $r_i\sim$ Lap($\frac{1}{\varepsilon_i^{'}}$) and $\varepsilon_i^{'} = \log\left(\frac{exp(\varepsilon)-w_i}{

Figures (5)

  • Figure 1: Our proposed DP-MLD for FoG detection: In multimodal representative learning scheme (first three rows), process EEG using BERT as text and other modalities using ViT as images, with cross-attention decoders for feature extraction; In feature-level privacy protection scheme (last row), apply a feature-level Laplacian dropout under DP scheme, optimizing randomness allocation between dropout and Laplacian noise to maximize performance and privacy protection.
  • Figure 2: Accuracy and loss in training and testing with different privacy budgets.
  • Figure 3: Best accuracy with different privacy budgets in testing.
  • Figure 4: Accuracy in training and testing across epochs for different privacy budgets.
  • Figure 5: Dropout rate, Laplacian noise scale and feature magnitude of EEG features (first row), OM features (second row), and CM features (third row) features.

Theorems & Definitions (5)

  • Definition 1
  • Definition 2
  • Definition 3
  • Theorem 1
  • Proof 1