Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Spiking Personalized Federated Learning for Brain-Computer Interface-Enabled Immersive Communication

Chen Shang, Dinh Thai Hoang, Diep N. Nguyen, Jiadong Yu

Abstract

This work proposes a novel immersive communication framework that leverages brain-computer interface (BCI) to acquire brain signals for inferring user-centric states (e.g., intention and perception-related discomfort), thereby enabling more personalized and robust immersive adaptation under strong individual variability. Specifically, we develop a personalized federated learning (PFL) model to analyze and process the collected brain signals, which not only accommodates neurodiverse brain-signal data but also prevents the leakage of sensitive brain-signal information. To address the energy bottleneck of continual on-device learning and inference on energy-limited immersive terminals (e.g., head-mounted display), we further embed spiking neural networks (SNNs) into the PFL. By exploiting sparse, event-driven spike computation, the SNN-enabled PFL reduces the computation and energy cost of training and inference while maintaining competitive personalization performance. Experiments on real brain-signal dataset demonstrate that our method achieves the best overall identification accuracy while reducing inference energy by 6.46$\times$ compared with conventional artificial neural network-based personalized baselines.

Spiking Personalized Federated Learning for Brain-Computer Interface-Enabled Immersive Communication

Abstract

This work proposes a novel immersive communication framework that leverages brain-computer interface (BCI) to acquire brain signals for inferring user-centric states (e.g., intention and perception-related discomfort), thereby enabling more personalized and robust immersive adaptation under strong individual variability. Specifically, we develop a personalized federated learning (PFL) model to analyze and process the collected brain signals, which not only accommodates neurodiverse brain-signal data but also prevents the leakage of sensitive brain-signal information. To address the energy bottleneck of continual on-device learning and inference on energy-limited immersive terminals (e.g., head-mounted display), we further embed spiking neural networks (SNNs) into the PFL. By exploiting sparse, event-driven spike computation, the SNN-enabled PFL reduces the computation and energy cost of training and inference while maintaining competitive personalization performance. Experiments on real brain-signal dataset demonstrate that our method achieves the best overall identification accuracy while reducing inference energy by 6.46 compared with conventional artificial neural network-based personalized baselines.
Paper Structure (20 sections, 2 theorems, 15 equations, 3 figures)

This paper contains 20 sections, 2 theorems, 15 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Preliminaries and System Overview
  3. Brain-Computer Interface and Brain Signal Process
  4. System Overview
  5. Proposed Learning Algorithm
  6. Personalized Federated Learning
  7. Energy-efficient Personalized FL-enabled by SNN
  8. SNN Framework
  9. Training of SNN-based Personalized FL
  10. Convergence Analysis and Benefits of SNN-based PFL
  11. Convergence analysis
  12. SNN sparsity for drift mitigation under neurodiversity
  13. Energy efficiency enabled by sparse spiking activity
  14. Performance Evaluations
  15. Experimental Settings
  16. ...and 5 more sections

Key Result

Theorem 1

Consider the above PFL procedure that, at each global round $r$, performs $E$ local updates of the form eq:prox_sgd_step to approximately solve eq:local_prox_sur, followed by weighted aggregation to update $\boldsymbol{w}^{(r)}$. Under standard step-size conditions required by proximal-PFL analyses where $\tilde{F}_{\mu}(\boldsymbol{w})$ is the surrogate envelope (i.e., the surrogate Moreau envel

Figures (3)

  • Figure 1: Illustration of the proposed BCI-enabled immersive communication framework supported by SNN-driven PFL. A wireless edge server (WES) hosts immersive applications and delivers VR/AR experiences to $K$ users. Each user is equipped with an integrated BCI-HMD that enables simultaneous immersive interaction and brain-signal acquisition. The acquired brain signals are processed locally by an SNN with sparse, event-driven spiking activity, enabling energy-efficient inference and continual personalization.
  • Figure 2: Brain signal identification accuracy vs. training round for various methods.
  • Figure 3: Energy consumption of various methods during model inference.

Theorems & Definitions (4)

  • Theorem 1
  • proof
  • Proposition 1: Sparsity-induced drift upper bound
  • proof