Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Compute-Update Federated Learning: A Lattice Coding Approach Over-the-Air

Seyed Mohammad Azimi-Abarghouyi, Lav R. Varshney

TL;DR

This paper introduces a federated learning framework that enables over-the-air computation via digital communications, using a new joint source-channel coding scheme that consistently delivers superior learning accuracy across various parameters and markedly surpasses other over-the-air methodologies.

Abstract

This paper introduces a federated learning framework that enables over-the-air computation via digital communications, using a new joint source-channel coding scheme. Without relying on channel state information at devices, this scheme employs lattice codes to both quantize model parameters and exploit interference from the devices. We propose a novel receiver structure at the server, designed to reliably decode an integer combination of the quantized model parameters as a lattice point for the purpose of aggregation. We present a mathematical approach to derive a convergence bound for the proposed scheme and offer design remarks. In this context, we suggest an aggregation metric and a corresponding algorithm to determine effective integer coefficients for the aggregation in each communication round. Our results illustrate that, regardless of channel dynamics and data heterogeneity, our scheme consistently delivers superior learning accuracy across various parameters and markedly surpasses other over-the-air methodologies.

Compute-Update Federated Learning: A Lattice Coding Approach Over-the-Air

TL;DR

This paper introduces a federated learning framework that enables over-the-air computation via digital communications, using a new joint source-channel coding scheme that consistently delivers superior learning accuracy across various parameters and markedly surpasses other over-the-air methodologies.

Abstract

This paper introduces a federated learning framework that enables over-the-air computation via digital communications, using a new joint source-channel coding scheme. Without relying on channel state information at devices, this scheme employs lattice codes to both quantize model parameters and exploit interference from the devices. We propose a novel receiver structure at the server, designed to reliably decode an integer combination of the quantized model parameters as a lattice point for the purpose of aggregation. We present a mathematical approach to derive a convergence bound for the proposed scheme and offer design remarks. In this context, we suggest an aggregation metric and a corresponding algorithm to determine effective integer coefficients for the aggregation in each communication round. Our results illustrate that, regardless of channel dynamics and data heterogeneity, our scheme consistently delivers superior learning accuracy across various parameters and markedly surpasses other over-the-air methodologies.
Paper Structure (17 sections, 4 theorems, 80 equations, 11 figures, 1 table)

This paper contains 17 sections, 4 theorems, 80 equations, 11 figures, 1 table.

Table of Contents

  1. Introduction
  2. Prior Work
  3. Key Contributions
  4. System Model
  5. Setup
  6. Learning Algorithm
  7. Lattice Preliminaries
  8. FedCPU: Compute-Update Scheme
  9. Transmission Scheme
  10. Aggregation Scheme
  11. First Layer
  12. Second Layer
  13. Convergence Analysis
  14. Integer Coefficient Selection
  15. Experimental Results
  16. ...and 2 more sections

Key Result

Theorem 1

The optimal equalization vector for a given coefficient vector $\mathbf{a}$ is

Figures (11)

  • Figure 1: Compute-update FL system.
  • Figure 2: Transmitter structure for device $k$.
  • Figure 3: Receiver structure for the server.
  • Figure 4: Two-dimensional hexagonal lattice example, with a scenario including two devices. The shown green point $3 \overline{\Delta\mathbf{w}}_{1}+\overline{\Delta\mathbf{w}}_2- 3\mathbf{d}_1 - \mathbf{d}_2$ is processed for aggregation in the form of $\Delta\mathbf{w}_\text{G} = \frac{3 \overline{\Delta\mathbf{w}}_{1}+\overline{\Delta\mathbf{w}}_2- 3\mathbf{d}_1 - \mathbf{d}_2}{4\eta}+\frac{3 \vartheta_1 + \vartheta_2}{4}\mathbf{1}$, as in \ref{['model_estimate']}.
  • Figure 5: MNIST, i.i.d. case.
  • ...and 6 more figures

Theorems & Definitions (15)

  • Definition 1
  • Definition 2
  • Definition 3
  • Definition 4
  • Theorem 1
  • Remark 1
  • Remark 2
  • Theorem 2
  • Theorem 3
  • Remark 3
  • ...and 5 more