Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Federated Learning of Binary Neural Networks: Enabling Low-Cost Inference

Nitin Priyadarshini Shankar, Soham Lahiri, Sheetal Kalyani, Saurav Prakash

Abstract

Federated Learning (FL) preserves privacy by distributing training across devices. However, using DNNs is computationally intensive at the low-powered edge during inference. Edge deployment demands models that simultaneously optimize memory footprint and computational efficiency, a dilemma where conventional DNNs fail by exceeding resource limits. Traditional post-training binarization reduces model size but suffers from severe accuracy loss due to quantization errors. To address these challenges, we propose FedBNN, a rotation-aware binary neural network framework that learns binary representations directly during local training. By encoding each weight as a single bit $\{+1, -1\}$ instead of a $32$-bit float, FedBNN shrinks the model footprint, significantly reducing runtime (during inference) FLOPs and memory requirements in comparison to federated methods using real models. Evaluations across multiple benchmark datasets demonstrate that FedBNN significantly reduces resource consumption while performing similarly to existing federated methods using real-valued models.

Federated Learning of Binary Neural Networks: Enabling Low-Cost Inference

Abstract

Federated Learning (FL) preserves privacy by distributing training across devices. However, using DNNs is computationally intensive at the low-powered edge during inference. Edge deployment demands models that simultaneously optimize memory footprint and computational efficiency, a dilemma where conventional DNNs fail by exceeding resource limits. Traditional post-training binarization reduces model size but suffers from severe accuracy loss due to quantization errors. To address these challenges, we propose FedBNN, a rotation-aware binary neural network framework that learns binary representations directly during local training. By encoding each weight as a single bit instead of a -bit float, FedBNN shrinks the model footprint, significantly reducing runtime (during inference) FLOPs and memory requirements in comparison to federated methods using real models. Evaluations across multiple benchmark datasets demonstrate that FedBNN significantly reduces resource consumption while performing similarly to existing federated methods using real-valued models.
Paper Structure (27 sections, 23 equations, 13 figures, 6 tables, 1 algorithm)

This paper contains 27 sections, 23 equations, 13 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Federated Learning
  4. Binarized Neural Network
  5. Proposed Method - FedBNN
  6. Proposed Rotation-Aware Client Training
  7. Trainable rotation weight with global memory
  8. Adjustable rotated weight vector for federated learning with global memory
  9. Training aware approximation for federated learning
  10. Proposed Aggregation Strategy
  11. Runtime computation savings for binary models
  12. Experimental Evaluation
  13. Setup
  14. Datasets and Partitioning
  15. Simulation results
  16. ...and 12 more sections

Figures (13)

  • Figure 1: FedBNN overall architecture.
  • Figure 2: Performance comparison between FedAvg and FedBNN on FMNIST using the CNN4 architecture under IID, Non-IID1 (Dirichlet $\alpha=0.3$), and Non-IID2 (label-skew) data partitions.
  • Figure 3: Performance comparison between FedAvg and FedBNN on SVHN using the CNN4 architecture under IID, Non-IID1 (Dirichlet $\alpha=0.3$), and Non-IID2 (label-skew) data partitions.
  • Figure 4: Performance comparison between FedAvg and FedBNN on CIFAR10 using the ResNet10 architecture under IID, Non-IID1 (Dirichlet $\alpha=0.3$), and Non-IID2 (label-skew) data partitions.
  • Figure 5: Performance comparison between FedAvg and FedBNN on CIFAR10 using the ConvNeXt-Tiny architecture under IID, Non-IID1 (Dirichlet $\alpha=0.3$), and Non-IID2 (label-skew) data partitions.
  • ...and 8 more figures