FedAIoT: A Federated Learning Benchmark for Artificial Intelligence of Things

TL;DR

FedAIoT presents the first federated learning benchmark tailored for AIoT by assembling eight authentic IoT datasets that cover diverse modalities and applications. It offers a unified end-to-end FL framework that includes IoT-specific preprocessing, non-IID data partitioning schemes, IoT-friendly models, and an IoT-factor emulator to simulate label noise and quantization. The benchmark enables systematic analysis of data heterogeneity, sampling strategies, label noise, and quantized training, revealing both opportunities and challenges for FL in IoT contexts. Overall, FedAIoT provides a practical, extensible resource to advance FL research for real-world AIoT deployments and emphasizes resilience, efficiency, and realistic evaluation.

Abstract

There is a significant relevance of federated learning (FL) in the realm of Artificial Intelligence of Things (AIoT). However, most existing FL works do not use datasets collected from authentic IoT devices and thus do not capture unique modalities and inherent challenges of IoT data. To fill this critical gap, in this work, we introduce FedAIoT, an FL benchmark for AIoT. FedAIoT includes eight datasets collected from a wide range of IoT devices. These datasets cover unique IoT modalities and target representative applications of AIoT. FedAIoT also includes a unified end-to-end FL framework for AIoT that simplifies benchmarking the performance of the datasets. Our benchmark results shed light on the opportunities and challenges of FL for AIoT. We hope FedAIoT could serve as an invaluable resource to foster advancements in the important field of FL for AIoT. The repository of FedAIoT is maintained at https://github.com/AIoT-MLSys-Lab/FedAIoT.

Figures (2)

• Figure 1: Overview of FedAIoT.
• Figure 2: Overview of the end-to-end FL framework for AIoT included in FedAIoT.