BrainGuard: Privacy-Preserving Multisubject Image Reconstructions from Brain Activities

TL;DR

BrainGuard tackles the challenge of reconstructing perceived images from multisubject fMRI data under strict privacy constraints by introducing a privacy-preserving global-local training framework. The method trains per-subject local models alongside a shared global model in a single multisubject session, using EMA-based smoothing and a novel hybrid synchronization that preserves foundational subject-specific signals while aligning intermediate features and adaptively fusing high-level representations via a Dynamic Fusion Learner. Training optimizes CLIP-aligned embeddings with MSE and SoftCLIP losses, guiding diffusion-based image reconstruction, and achieves state-of-the-art performance on the NSD dataset with a single, privacy-preserving training run. The work demonstrates significant potential for privacy-preserving brain decoding, improving reconstruction quality while maintaining data confidentiality, and outlines directions for extending to other modalities and brain regions.

Abstract

Reconstructing perceived images from human brain activity forms a crucial link between human and machine learning through Brain-Computer Interfaces. Early methods primarily focused on training separate models for each individual to account for individual variability in brain activity, overlooking valuable cross-subject commonalities. Recent advancements have explored multisubject methods, but these approaches face significant challenges, particularly in data privacy and effectively managing individual variability. To overcome these challenges, we introduce BrainGuard, a privacy-preserving collaborative training framework designed to enhance image reconstruction from multisubject fMRI data while safeguarding individual privacy. BrainGuard employs a collaborative global-local architecture where individual models are trained on each subject's local data and operate in conjunction with a shared global model that captures and leverages cross-subject patterns. This architecture eliminates the need to aggregate fMRI data across subjects, thereby ensuring privacy preservation. To tackle the complexity of fMRI data, BrainGuard integrates a hybrid synchronization strategy, enabling individual models to dynamically incorporate parameters from the global model. By establishing a secure and collaborative training environment, BrainGuard not only protects sensitive brain data but also improves the image reconstructions accuracy. Extensive experiments demonstrate that BrainGuard sets a new benchmark in both high-level and low-level metrics, advancing the state-of-the-art in brain decoding through its innovative design.

Figures (7)

• Figure 1: (a) Early subject-specific methods require separate training for each individual using their respective fMRI, overlooking intersubject commonalities. (b) Recent multisubject methods that combine all subjects' fMRI for training pose substantial privacy concerns. (c) BrainGuard captures intersubject commonalities while preserving data privacy.
• Figure 2: (a) t-SNE visualization of the embeddings output from the subject-specific layers in MindBridge wang2024mindbridge. (b) represents ones from BrainGuard models. Both are visualized upon the NSD allen2022massivetest set.
• Figure 3: An overview of the BrainGuard training and inference framework (§\ref{['sec_framework']}).
• Figure 4: Qualitative comparisons on the NSD test dataset.BrainGuard performs a single training session on multisubject fMRI data, demonstrates superior reconstruction accuracy compared to four recent state-of-the-art methods quan2024psychometrywang2024mindbridgescotti2023reconstructingozcelik2023brain, while effectively preserving data privacy.
• Figure 5: (a) We evaluate our trained global model and compare it with the SOTA (MindBridge). (b) Performance changes with different numbers of participating subjects.