Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Communication-Efficient Federated Learning via Clipped Uniform Quantization

Zavareh Bozorgasl, Hao Chen

TL;DR

Addresses the costly uplink in federated learning by proposing clipped uniform quantization with an analytically derived symmetric clipping threshold $s$ that minimizes $E[(X-Q(X))^2]$, paired with stochastic quantization to reduce bias. The OCTAV framework uses quantization-aware training and two aggregation rules: FedAvg weights $\frac{n_i}{N}$ and inverse-average quantization-error weights $\frac{1/e_{ij}}{\sum_j 1/e_{ij}}$, with the global objective $f(z)=\sum_{i=1}^M \frac{n_i}{N} f_i(z)$. Contributions include the derivation of the optimal clipping threshold, a comparison of stochastic versus deterministic quantization, and a new error-based aggregation strategy validated on MNIST and CIFAR-10, which achieves near-full-precision performance with substantial uplink savings and privacy advantages. The work highlights practical gains for scalable, privacy-preserving federated learning on resource-constrained devices and motivates future directions in per-layer or per-channel bit-width quantization and activation/gradient quantization.

Abstract

This paper presents a novel approach to enhance communication efficiency in federated learning through clipped uniform quantization. By leveraging optimal clipping thresholds and client-specific adaptive quantization schemes, the proposed method significantly reduces bandwidth and memory requirements for model weight transmission between clients and the server while maintaining competitive accuracy. We investigate the effects of symmetric clipping and uniform quantization on model performance, emphasizing the role of stochastic quantization in mitigating artifacts and improving robustness. Extensive simulations demonstrate that the method achieves near-full-precision performance with substantial communication savings. Moreover, the proposed approach facilitates efficient weight averaging based on the inverse of the mean squared quantization errors, effectively balancing the trade-off between communication efficiency and model accuracy. Moreover, in contrast to federated averaging, this design obviates the need to disclose client-specific data volumes to the server, thereby enhancing client privacy. Comparative analysis with conventional quantization methods further confirms the efficacy of the proposed scheme.

Communication-Efficient Federated Learning via Clipped Uniform Quantization

TL;DR

Addresses the costly uplink in federated learning by proposing clipped uniform quantization with an analytically derived symmetric clipping threshold that minimizes , paired with stochastic quantization to reduce bias. The OCTAV framework uses quantization-aware training and two aggregation rules: FedAvg weights and inverse-average quantization-error weights , with the global objective . Contributions include the derivation of the optimal clipping threshold, a comparison of stochastic versus deterministic quantization, and a new error-based aggregation strategy validated on MNIST and CIFAR-10, which achieves near-full-precision performance with substantial uplink savings and privacy advantages. The work highlights practical gains for scalable, privacy-preserving federated learning on resource-constrained devices and motivates future directions in per-layer or per-channel bit-width quantization and activation/gradient quantization.

Abstract

This paper presents a novel approach to enhance communication efficiency in federated learning through clipped uniform quantization. By leveraging optimal clipping thresholds and client-specific adaptive quantization schemes, the proposed method significantly reduces bandwidth and memory requirements for model weight transmission between clients and the server while maintaining competitive accuracy. We investigate the effects of symmetric clipping and uniform quantization on model performance, emphasizing the role of stochastic quantization in mitigating artifacts and improving robustness. Extensive simulations demonstrate that the method achieves near-full-precision performance with substantial communication savings. Moreover, the proposed approach facilitates efficient weight averaging based on the inverse of the mean squared quantization errors, effectively balancing the trade-off between communication efficiency and model accuracy. Moreover, in contrast to federated averaging, this design obviates the need to disclose client-specific data volumes to the server, thereby enhancing client privacy. Comparative analysis with conventional quantization methods further confirms the efficacy of the proposed scheme.
Paper Structure (8 sections, 7 equations, 9 figures, 3 tables, 1 algorithm)

This paper contains 8 sections, 7 equations, 9 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Proposed Federated Learning Algorithm
  3. Clipped Uniform Quantizers
  4. Clipping and two schemes of quantization
  5. Example: Stochastic vs Deterministic Quantization
  6. Averaging based on average squared quantization errors of weights
  7. Simulations Results
  8. Conclusion

Figures (9)

  • Figure 1: An illustration of a typical federated learning system.
  • Figure 2: Communication rounds (Epoch) versus training and test accuracy (percent) for MNIST dataset.
  • Figure 3: Communication rounds (Epoch) versus training and test accuracy (percent) for MNIST dataset for different number of clients.
  • Figure 4: Communication rounds (Epoch) versus training and test accuracy (percent) for CIFAR10 dataset.
  • Figure 5: Comparison of training and test accuracy (percentage) across communication rounds (epochs) for different neural network configurations on the CIFAR-10 dataset.
  • ...and 4 more figures