Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Robustness of Spiking Neural Networks in Federated Learning with Compression Against Non-omniscient Byzantine Attacks

Manh V. Nguyen, Liang Zhao, Bobin Deng, Shaoen Wu

TL;DR

The paper addresses robust, energy-efficient on-device learning by evaluating Spiking Neural Networks (SNNs) in Federated Learning (FL) under non-omniscient Byzantine attacks and bandwidth constraints. It shows that integrating gradient compression via $ Top-\kappa $ sparsification exploits SNNs' robustness to improve both communication efficiency and attack resilience, outperforming FL-ANNs in most scenarios. Specifically, under the MinMax attack, FL-SNNs with compression yield up to a $40\%$ accuracy gain, while bandwidth reductions reach approximately $33\times$ compared to FL-ANNs. These findings highlight the practical potential of FL-SNNs with compression for energy-efficient, robust edge AI and inform future defense mechanisms and system design.

Abstract

Spiking Neural Networks (SNNs), which offer exceptional energy efficiency for inference, and Federated Learning (FL), which offers privacy-preserving distributed training, is a rising area of interest that highly beneficial towards Internet of Things (IoT) devices. Despite this, research that tackles Byzantine attacks and bandwidth limitation in FL-SNNs, both poses significant threats on model convergence and training times, still remains largely unexplored. Going beyond proposing a solution for both of these problems, in this work we highlight the dual benefits of FL-SNNs, against non-omniscient Byzantine adversaries (ones that restrict attackers access to local clients datasets), and greater communication efficiency, over FL-ANNs. Specifically, we discovered that a simple integration of Top-\k{appa} sparsification into the FL apparatus can help leverage the advantages of the SNN models in both greatly reducing bandwidth usage and significantly boosting the robustness of FL training against non-omniscient Byzantine adversaries. Most notably, we saw a massive improvement of roughly 40% accuracy gain in FL-SNNs training under the lethal MinMax attack

The Robustness of Spiking Neural Networks in Federated Learning with Compression Against Non-omniscient Byzantine Attacks

TL;DR

The paper addresses robust, energy-efficient on-device learning by evaluating Spiking Neural Networks (SNNs) in Federated Learning (FL) under non-omniscient Byzantine attacks and bandwidth constraints. It shows that integrating gradient compression via sparsification exploits SNNs' robustness to improve both communication efficiency and attack resilience, outperforming FL-ANNs in most scenarios. Specifically, under the MinMax attack, FL-SNNs with compression yield up to a accuracy gain, while bandwidth reductions reach approximately compared to FL-ANNs. These findings highlight the practical potential of FL-SNNs with compression for energy-efficient, robust edge AI and inform future defense mechanisms and system design.

Abstract

Spiking Neural Networks (SNNs), which offer exceptional energy efficiency for inference, and Federated Learning (FL), which offers privacy-preserving distributed training, is a rising area of interest that highly beneficial towards Internet of Things (IoT) devices. Despite this, research that tackles Byzantine attacks and bandwidth limitation in FL-SNNs, both poses significant threats on model convergence and training times, still remains largely unexplored. Going beyond proposing a solution for both of these problems, in this work we highlight the dual benefits of FL-SNNs, against non-omniscient Byzantine adversaries (ones that restrict attackers access to local clients datasets), and greater communication efficiency, over FL-ANNs. Specifically, we discovered that a simple integration of Top-\k{appa} sparsification into the FL apparatus can help leverage the advantages of the SNN models in both greatly reducing bandwidth usage and significantly boosting the robustness of FL training against non-omniscient Byzantine adversaries. Most notably, we saw a massive improvement of roughly 40% accuracy gain in FL-SNNs training under the lethal MinMax attack
Paper Structure (20 sections, 3 equations, 7 figures)

This paper contains 20 sections, 3 equations, 7 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Background
  4. Spiking Neural Networks (SNNs)
  5. Federated Learning (FL)
  6. Non-omniscient Byzantine Adversaries
  7. The Byzantine methodology
  8. The non-omniscient attacks
  9. Pre-Evaluation: FL-SNNs under Non-omniscient Byzantine & Top-$\kappa$ Sparsification
  10. Non-omniscient Byzantine Adversaries
  11. Experiments
  12. Observation
  13. Federated Learning with Top-$\kappa$ Sparsification
  14. Compression mechanism
  15. Prospects
  16. ...and 5 more sections

Figures (7)

  • Figure 1: The Leaky Integrate & Fire (LIF) Spiking Neuron.
  • Figure 2: FL-SNNs Training under Non-omniscient Byzantine Adversary.
  • Figure 3: The robustness of FL-SNNs and FL-ANNs against 4 randomized-noise-based Byzantine adversaries. (Note: Loss bars are not to be confused with the training loss, but $\textit{Loss} = \textit{Clean Accuracy} - \textit{Attacked Accuracy}$)
  • Figure 4: Illustration of FL with Top-$\kappa$ sparsification
  • Figure 5: a) Total bandwidth consumption of FL-SNN and FL-ANN w/ and w/o compression; b) Gradients' frequency of Top-$\kappa$ retention heatmap.
  • ...and 2 more figures

Theorems & Definitions (2)

  • Remark 1
  • Remark 2