Timely Parameter Updating in Over-the-Air Federated Learning

TL;DR

This work tackles the dimension–waveform mismatch in over-the-air Federated Learning by introducing FAIR-k, an age-aware gradient updating policy that selects a small, impactful subset of gradient entries to update per round. FAIR-k blends magnitude-based Top-k selection with an AoU-driven freshness criterion, and its dynamics are analyzed via a Markov-chain model to quantify parameter staleness. The authors derive a convergence-rate bound that exposes how data heterogeneity, channel noise, and update freshness jointly affect training efficiency, and they demonstrate superior performance through extensive simulations and a hardware SDR prototype. The results show that FAIR-k accelerates convergence and reduces communication overhead without sacrificing training accuracy, highlighting the practical viability of freshness-aware updates in resource-constrained OAC-FL systems.

Abstract

Incorporating over-the-air computations (OAC) into the model training process of federated learning (FL) is an effective approach to alleviating the communication bottleneck in FL systems. Under OAC-FL, every client modulates its intermediate parameters, such as gradient, onto the same set of orthogonal waveforms and simultaneously transmits the radio signal to the edge server. By exploiting the superposition property of multiple-access channels, the edge server can obtain an automatically aggregated global gradient from the received signal. However, the limited number of orthogonal waveforms available in practical systems is fundamentally mismatched with the high dimensionality of modern deep learning models. To address this issue, we propose Freshness Freshness-mAgnItude awaRe top-k (FAIR-k), an algorithm that selects, in each communication round, the most impactful subset of gradients to be updated over the air. In essence, FAIR-k combines the complementary strengths of the Round-Robin and Top-k algorithms, striking a delicate balance between timeliness (freshness of parameter updates) and importance (gradient magnitude). Leveraging tools from Markov analysis, we characterize the distribution of parameter staleness under FAIR-k. Building on this, we establish the convergence rate of OAC-FL with FAIR-k, which discloses the joint effect of data heterogeneity, channel noise, and parameter staleness on the training efficiency. Notably, as opposed to conventional analyses that assume a universal Lipschitz constant across all the clients, our framework adopts a finer-grained model of the data heterogeneity. The analysis demonstrates that since FAIR-k promotes fresh (and fair) parameter updates, it not only accelerates convergence but also enhances communication efficiency by enabling an extended period of local training without significantly affecting overall training efficiency.

Figures (10)

• Figure 1: An overview of the edge learning system. The following steps are repeated until convergence: (1) each client calculates the local gradient based on its local dataset and uploads the compressed gradient to the server via analog transmissions; (2) the server extracts an automatically aggregated global gradient from the received radio signal, and reconstructs it to update the global model; (3) the updated model and selection vector are broadcast to all the clients for a new round of local updating.
• Figure 2: Visual illustrations of the selection process for FAIR-$k$ and the dynamics of entry position transition, modeled as a Markov chain.
• Figure 3: The distribution of AoU.
• Figure 4: Performance comparison for test accuracy. Here, (a) and (b) result from training ResNet-18 on the CIFAR-10 dataset with i.i.d. and non-i.i.d. partitions, respectively, while (c) and (d) result from training ResNet-18 on the CIFAR-100 dataset with i.i.d. and non-i.i.d. partitions, respectively.
• Figure 5: The impact of parameter updating schemes on parameter staleness, exemplified by training ResNet-18 on the CIFAR-10 dataset.