Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Federated Learning with Multi-resolution Model Broadcast

Henrik Rydén, Reza Moosavi, Erik G. Larsson

TL;DR

Federated learning requires downlink distribution of the global model, which is challenged by heterogeneous wireless channels across agents. The authors propose a downlink strategy based on multi-resolution modulation that splits the model into a coarse low-resolution part and a high-resolution refinement for better channels, and uses differential updates across communication rounds. They validate the approach with MNIST experiments showing that mixed-resolution broadcasting outperforms uniform low-resolution and approaches full-resolution performance, and discuss extensions such as beamforming, multicast grouping, and integration with 5G NR for model broadcast. The results indicate improved downlink resource efficiency, faster iteration times, and greater participation from devices with weak downlink SNR, enabling more robust wireless federated learning.

Abstract

In federated learning, a server must periodically broadcast a model to the agents. We propose to use multi-resolution coding and modulation (also known as non-uniform modulation) for this purpose. In the simplest instance, broadcast transmission is used, whereby all agents are targeted with one and the same transmission (typically without any particular favored beam direction), which is coded using multi-resolution coding/modulation. This enables high-SNR agents, with high path gains to the server, to receive a more accurate model than the low-SNR agents do, without consuming more downlink resources. As one implementation, we use transmission with a non-uniform 8-PSK constellation, where a high-SNR receiver (agent) can separate all 8 constellation points (hence receive 3 bits) whereas a low-SNR receiver can only separate 4 points (hence receive 2 bits). By encoding the least significant information in the third bit, the high-SNR receivers can obtain the model with higher accuracy, while the low-SNR receiver can still obtain the model although with reduced accuracy, thereby facilitating at least some basic participation of the low-SNR receiver. We show the effectiveness of our proposed scheme via experimentation using federated learning with the MNIST data-set.

Federated Learning with Multi-resolution Model Broadcast

TL;DR

Federated learning requires downlink distribution of the global model, which is challenged by heterogeneous wireless channels across agents. The authors propose a downlink strategy based on multi-resolution modulation that splits the model into a coarse low-resolution part and a high-resolution refinement for better channels, and uses differential updates across communication rounds. They validate the approach with MNIST experiments showing that mixed-resolution broadcasting outperforms uniform low-resolution and approaches full-resolution performance, and discuss extensions such as beamforming, multicast grouping, and integration with 5G NR for model broadcast. The results indicate improved downlink resource efficiency, faster iteration times, and greater participation from devices with weak downlink SNR, enabling more robust wireless federated learning.

Abstract

In federated learning, a server must periodically broadcast a model to the agents. We propose to use multi-resolution coding and modulation (also known as non-uniform modulation) for this purpose. In the simplest instance, broadcast transmission is used, whereby all agents are targeted with one and the same transmission (typically without any particular favored beam direction), which is coded using multi-resolution coding/modulation. This enables high-SNR agents, with high path gains to the server, to receive a more accurate model than the low-SNR agents do, without consuming more downlink resources. As one implementation, we use transmission with a non-uniform 8-PSK constellation, where a high-SNR receiver (agent) can separate all 8 constellation points (hence receive 3 bits) whereas a low-SNR receiver can only separate 4 points (hence receive 2 bits). By encoding the least significant information in the third bit, the high-SNR receivers can obtain the model with higher accuracy, while the low-SNR receiver can still obtain the model although with reduced accuracy, thereby facilitating at least some basic participation of the low-SNR receiver. We show the effectiveness of our proposed scheme via experimentation using federated learning with the MNIST data-set.
Paper Structure (12 sections, 3 equations, 2 figures, 1 table)

This paper contains 12 sections, 3 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Contributions
  3. Related Work
  4. Proposed Approach
  5. Non-uniform modulation
  6. Partitioning of the learning model
  7. Partitioning of differential update of learning model
  8. Numerical Example
  9. Simulation setting
  10. Simulation results
  11. Extensions
  12. Concluding Remarks

Figures (2)

  • Figure 1: Example of nonuniform constellation for a single-antenna transmitter.
  • Figure 2: Classification accuracy results for the MNIST test dataset. The model is trained using FL with 4 agents that experience different resolution when receiving the NN parameters.