Redefining Clustered Federated Learning for System Identification: The Path of ClusterCraft
Ertuğrul Keçeci, Müjde Güzelkaya, Tufan Kumbasar
TL;DR
The paper tackles SYSID in a federated setting with heterogeneous data by proposing IC-SYSID, a framework that combines Incremental Clustering (ClusterCraft) with ClusterMerge and an enhanced variant (eCC) to learn multiple stable cluster models without prior dataset knowledge. It introduces stability-focused regularization and scaled Glorot initialization, plus mini-batch deep learning to handle large local datasets. Empirical results on a fleet-of-vehicles scenario show IC-SYSID achieves higher predictive/simulative accuracy while markedly reducing unstable cluster models and clustering overhead compared to C-SYSID. The work advances practical clustered FL for SYSID, enabling robust, scalable learning across distributed, diverse data sources.
Abstract
This paper addresses the System Identification (SYSID) problem within the framework of federated learning. We introduce a novel algorithm, Incremental Clustering-based federated learning method for SYSID (IC-SYSID), designed to tackle SYSID challenges across multiple data sources without prior knowledge. IC-SYSID utilizes an incremental clustering method, ClusterCraft (CC), to eliminate the dependency on the prior knowledge of the dataset. CC starts with a single cluster model and assigns similar local workers to the same clusters by dynamically increasing the number of clusters. To reduce the number of clusters generated by CC, we introduce ClusterMerge, where similar cluster models are merged. We also introduce enhanced ClusterCraft to reduce the generation of similar cluster models during the training. Moreover, IC-SYSID addresses cluster model instability by integrating a regularization term into the loss function and initializing cluster models with scaled Glorot initialization. It also utilizes a mini-batch deep learning approach to manage large SYSID datasets during local training. Through the experiments conducted on a real-world representing SYSID problem, where a fleet of vehicles collaboratively learns vehicle dynamics, we show that IC-SYSID achieves a high SYSID performance while preventing the learning of unstable clusters.