Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Robustness of Spiking Neural Networks in Communication and its Application towards Network Efficiency in Federated Learning

Manh V. Nguyen, Liang Zhao, Bobin Deng, William Severa, Honghui Xu, Shaoen Wu

TL;DR

This work investigates the robustness of Spiking Neural Networks (SNNs) in Federated Learning (FL) under noisy communication and develops two compression-based strategies to curb bandwidth: Top-$\\kappa$ Sparsification (FLTS) and Dynamic-$\\kappa$ Reduction (FLDR). It demonstrates that SNNs are more resistant to transmission noise than equivalent ANNs and shows that substantial communication savings are achievable without sacrificing accuracy, with communicated parameters reduced to as low as $6\\%$ of the original. The proposed methods include a linear/exponential reduction scheme (and adaptive variants) that balance compression and convergence, validated on CIFAR-10 with a VGG9 SNN in a multi-client FL setting. Overall, the results highlight the practical potential of network-efficient FL using SNNs for edge devices, enabling scalable, low-bandwidth collaborative learning. The study contributes to robust FL design by revealing the resilience of SNNs to parameter perturbations and by delivering concrete, scalable sparsification and dynamic compression strategies.

Abstract

Spiking Neural Networks (SNNs) have recently gained significant interest in on-chip learning in embedded devices and emerged as an energy-efficient alternative to conventional Artificial Neural Networks (ANNs). However, to extend SNNs to a Federated Learning (FL) setting involving collaborative model training, the communication between the local devices and the remote server remains the bottleneck, which is often restricted and costly. In this paper, we first explore the inherent robustness of SNNs under noisy communication in FL. Building upon this foundation, we propose a novel Federated Learning with Top-K Sparsification (FLTS) algorithm to reduce the bandwidth usage for FL training. We discover that the proposed scheme with SNNs allows more bandwidth savings compared to ANNs without impacting the model's accuracy. Additionally, the number of parameters to be communicated can be reduced to as low as 6 percent of the size of the original model. We further improve the communication efficiency by enabling dynamic parameter compression during model training. Extensive experiment results demonstrate that our proposed algorithms significantly outperform the baselines in terms of communication cost and model accuracy and are promising for practical network-efficient FL with SNNs.

The Robustness of Spiking Neural Networks in Communication and its Application towards Network Efficiency in Federated Learning

TL;DR

This work investigates the robustness of Spiking Neural Networks (SNNs) in Federated Learning (FL) under noisy communication and develops two compression-based strategies to curb bandwidth: Top- Sparsification (FLTS) and Dynamic- Reduction (FLDR). It demonstrates that SNNs are more resistant to transmission noise than equivalent ANNs and shows that substantial communication savings are achievable without sacrificing accuracy, with communicated parameters reduced to as low as of the original. The proposed methods include a linear/exponential reduction scheme (and adaptive variants) that balance compression and convergence, validated on CIFAR-10 with a VGG9 SNN in a multi-client FL setting. Overall, the results highlight the practical potential of network-efficient FL using SNNs for edge devices, enabling scalable, low-bandwidth collaborative learning. The study contributes to robust FL design by revealing the resilience of SNNs to parameter perturbations and by delivering concrete, scalable sparsification and dynamic compression strategies.

Abstract

Spiking Neural Networks (SNNs) have recently gained significant interest in on-chip learning in embedded devices and emerged as an energy-efficient alternative to conventional Artificial Neural Networks (ANNs). However, to extend SNNs to a Federated Learning (FL) setting involving collaborative model training, the communication between the local devices and the remote server remains the bottleneck, which is often restricted and costly. In this paper, we first explore the inherent robustness of SNNs under noisy communication in FL. Building upon this foundation, we propose a novel Federated Learning with Top-K Sparsification (FLTS) algorithm to reduce the bandwidth usage for FL training. We discover that the proposed scheme with SNNs allows more bandwidth savings compared to ANNs without impacting the model's accuracy. Additionally, the number of parameters to be communicated can be reduced to as low as 6 percent of the size of the original model. We further improve the communication efficiency by enabling dynamic parameter compression during model training. Extensive experiment results demonstrate that our proposed algorithms significantly outperform the baselines in terms of communication cost and model accuracy and are promising for practical network-efficient FL with SNNs.
Paper Structure (19 sections, 8 equations, 8 figures, 1 table, 2 algorithms)

This paper contains 19 sections, 8 equations, 8 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background of Federated Learning and Spiking Neural Networks
  3. Spiking Neural Network
  4. Federated Learning
  5. SNN In-situ Training Hardware/System
  6. Robustness of SNN in FL with noisy communication
  7. Proposed Communication-efficient FL Algorithms with SNN
  8. Rationale
  9. Federated Learning with Top-$\kappa$ Sparsification
  10. Federated Learning with Dynamic-$\kappa$ Reduction
  11. Experimental Evaluation
  12. FL with Top-$\kappa$ Sparsification under Noisy Communication
  13. Effect of Compression Rate Parameter $\kappa$ in FLTS
  14. Impact of Network Size
  15. Effect of the Final Compression Rate $\omega$ in FLDR
  16. ...and 4 more sections

Figures (8)

  • Figure 1: Processing Mechanism of Integrate-and-Fire (IF) Spiking Neuron.
  • Figure 2: The stability of a) ANN and b) SNN under various noise levels $\sigma$ using absolute values.
  • Figure 3: The stability of a) ANN and b) SNN under various noise levels $\hat{\sigma}$ using relative percentage to the magnitude of the parameters.
  • Figure 4: Overview of Federated Learning with Spiking Neural Networks integrated with the proposed model compression algorithms.
  • Figure 5: Comparison of FL with Top-$\kappa$ Sparification under noisy communication between a) ANNs and b) SNNs.
  • ...and 3 more figures