Federated Learning-Distillation Alternation for Resource-Constrained IoT

TL;DR

This work tackles the energy and communication challenges of federated learning on resource-constrained IoT devices by introducing FL-distillation alternation (FLDA), which cycles between federated distillation (FD) and federated learning (FL) within a multichannel slotted-ALOHA energy-harvesting network under background traffic. The method derives an uplink-throughput framework that accounts for subpacketization, error-correcting codes, and Poisson background activity, and shows how alternating FD and FL can achieve higher accuracy than FD and faster convergence than FL while saving energy. A probabilistic network model links energy harvesting and channel access to active participation, enabling FLDA to adapt to varying energy-income and interference conditions; the approach is validated on MNIST with a non-IID six-layer CNN, demonstrating up to $98\%$ energy savings over FL and improved robustness to background traffic. Overall, FLDA offers a practical, low-overhead protocol that enhances resource efficiency and learning performance for IoT deployments with intermittent energy and congested wireless channels.

Abstract

Federated learning (FL) faces significant challenges in Internet of Things (IoT) networks due to device limitations in energy and communication resources, especially when considering the large size of FL models. From an energy perspective, the challenge is aggravated if devices rely on energy harvesting (EH), as energy availability can vary significantly over time, influencing the average number of participating users in each iteration. Additionally, the transmission of large model updates is more susceptible to interference from uncorrelated background traffic in shared wireless environments. As an alternative, federated distillation (FD) reduces communication overhead and energy consumption by transmitting local model outputs, which are typically much smaller than the entire model used in FL. However, this comes at the cost of reduced model accuracy. Therefore, in this paper, we propose FL-distillation alternation (FLDA). In FLDA, devices alternate between FD and FL phases, balancing model information with lower communication overhead and energy consumption per iteration. We consider a multichannel slotted-ALOHA EH-IoT network subject to background traffic/interference. In such a scenario, FLDA demonstrates higher model accuracy than both FL and FD, and achieves faster convergence than FL. Moreover, FLDA achieves target accuracies saving up to 98% in energy consumption, while also being less sensitive to interference, both relative to FL.

Figures (7)

• Figure 1: Distributed learning procedures: (a) FL iteration, (b) FD with $2$ possible labels.
• Figure 2: Communication system model. The illustrated scenario depicts devices performing FL, resulting in a frame upload window of $\tau_s F_{\text{FL}}$. If FD were employed instead, the frame upload window would be $\tau_s F_{\text{FD}}$, reflecting the smaller size of FD updates.
• Figure 3: Schematic of FLDA.
• Figure 4: Mean test accuracy as a function of time.
• Figure 5: Normalized average battery level as a function of time.