Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SFedHIFI: Fire Rate-Based Heterogeneous Information Fusion for Spiking Federated Learning

Ran Tao, Qiugang Zhan, Shantian Yang, Xiurui Xie, Qi Tian, Guisong Liu

Abstract

Spiking Federated Learning (SFL) has been widely studied with the energy efficiency of Spiking Neural Networks (SNNs). However, existing SFL methods require model homogeneity and assume all clients have sufficient computational resources, resulting in the exclusion of some resource-constrained clients. To address the prevalent system heterogeneity in real-world scenarios, enabling heterogeneous SFL systems that allow clients to adaptively deploy models of different scales based on their local resources is crucial. To this end, we introduce SFedHIFI, a novel Spiking Federated Learning framework with Fire Rate-Based Heterogeneous Information Fusion. Specifically, SFedHIFI employs channel-wise matrix decomposition to deploy SNN models of adaptive complexity on clients with heterogeneous resources. Building on this, the proposed heterogeneous information fusion module enables cross-scale aggregation among models of different widths, thereby enhancing the utilization of diverse local knowledge. Extensive experiments on three public benchmarks demonstrate that SFedHIFI can effectively enable heterogeneous SFL, consistently outperforming all three baseline methods. Compared with ANN-based FL, it achieves significant energy savings with only a marginal trade-off in accuracy.

SFedHIFI: Fire Rate-Based Heterogeneous Information Fusion for Spiking Federated Learning

Abstract

Spiking Federated Learning (SFL) has been widely studied with the energy efficiency of Spiking Neural Networks (SNNs). However, existing SFL methods require model homogeneity and assume all clients have sufficient computational resources, resulting in the exclusion of some resource-constrained clients. To address the prevalent system heterogeneity in real-world scenarios, enabling heterogeneous SFL systems that allow clients to adaptively deploy models of different scales based on their local resources is crucial. To this end, we introduce SFedHIFI, a novel Spiking Federated Learning framework with Fire Rate-Based Heterogeneous Information Fusion. Specifically, SFedHIFI employs channel-wise matrix decomposition to deploy SNN models of adaptive complexity on clients with heterogeneous resources. Building on this, the proposed heterogeneous information fusion module enables cross-scale aggregation among models of different widths, thereby enhancing the utilization of diverse local knowledge. Extensive experiments on three public benchmarks demonstrate that SFedHIFI can effectively enable heterogeneous SFL, consistently outperforming all three baseline methods. Compared with ANN-based FL, it achieves significant energy savings with only a marginal trade-off in accuracy.
Paper Structure (16 sections, 13 equations, 1 figure, 3 tables)

This paper contains 16 sections, 13 equations, 1 figure, 3 tables.

Table of Contents

  1. Introduction
  2. Preliminary and Related Work
  3. The Leaky Integrate-and-Fire Model
  4. Spiking Federated Learning
  5. System-Heterogeneous Federated Learning
  6. Method
  7. Heterogeneous FL Framework Based on Channel-Wise Matrix Decomposition
  8. Firing Rate-Based Heterogeneous Information Fusion (HIFI) Module
  9. The Local Objective Function
  10. Experiments
  11. Experimental Setup
  12. Benchmark Comparison Evaluation
  13. Ablation Experiment of the HIFI Module
  14. Energy Analysis
  15. Conclusion
  16. ...and 1 more sections

Figures (1)

  • Figure 1: Overview of SFedHIFI. First, local clients are clustered into the model scale set of computational resources $\mathbf{p}=\{0.25,0.5,075,1.0\}$. Then, the local $(\mathbf{p} \times)$ model gets the uniform shared basis $B$ and the model scale factor $M^{\mathbf{p}}$ through the Channel-wise Matrix Decomposition. In the central server, the uniform shared basis will be aggregated as the global $\overline{B}$, and the model scale factors will be aggregated into global $\{\overline{M}^{\mathbf{p}}\}$ corresponding to different model scales. The list of global model scale factors $\{\overline{M}^{\mathbf{p}}\}$ is then fed to the HIFI module, where the Firing Rate-Based Heterogeneous Information Fusion is applied to integrate knowledge across scales. This process enhances the representational capacity of global SNNs at all scales and improves their overall performance.