One-Shot Federated Clustering of Non-Independent Completely Distributed Data

TL;DR

This work addresses the challenge of federated clustering under a realistic Non-ICD setting where local clusters are incomplete and fragmented across clients. It introduces GOLD, a one-shot FC framework that combines Fine-grained Competitive Penalized Learning on clients with Multi-granular Competitive Penalized Learning on the server, augmented by Representation Enhancement via Multi-Granular Clusters (REMC) to fuse multi-granular information. The core contributions are the formalization of Non-ICD and a λ-based divergence measure, the FCPL and MCPL modules for robust micro- and multi-granular clustering, and REMC for integrating multi-granular distributions into a coherent global clustering with unknown k*. The approach achieves consistent improvements across ten datasets, exhibits robustness to varying Non-ICD degrees, and demonstrates scalable, privacy-conscious one-shot communication with interpretable nested cluster relationships, marking a practical advance for FC in heterogeneous edge environments.

Abstract

Federated Learning (FL) that extracts data knowledge while protecting the privacy of multiple clients has achieved remarkable results in distributed privacy-preserving IoT systems, including smart traffic flow monitoring, smart grid load balancing, and so on. Since most data collected from edge devices are unlabeled, unsupervised Federated Clustering (FC) is becoming increasingly popular for exploring pattern knowledge from complex distributed data. However, due to the lack of label guidance, the common Non-Independent and Identically Distributed (Non-IID) issue of clients have greatly challenged FC by posing the following problems: How to fuse pattern knowledge (i.e., cluster distribution) from Non-IID clients; How are the cluster distributions among clients related; and How does this relationship connect with the global knowledge fusion? In this paper, a more tricky but overlooked phenomenon in Non-IID is revealed, which bottlenecks the clustering performance of the existing FC approaches. That is, different clients could fragment a cluster, and accordingly, a more generalized Non-IID concept, i.e., Non-ICD (Non-Independent Completely Distributed), is derived. To tackle the above FC challenges, a new framework named GOLD (Global Oriented Local Distribution Learning) is proposed. GOLD first finely explores the potential incomplete local cluster distributions of clients, then uploads the distribution summarization to the server for global fusion, and finally performs local cluster enhancement under the guidance of the global distribution. Extensive experiments, including significance tests, ablation studies, scalability evaluations, qualitative results, etc., have been conducted to show the superiority of GOLD.

Figures (11)

• Figure 1: Non-ICD phenomenon and its impact on Federated Clustering (FC) performance. Most FC studies address Non-IID by assuming that the global optimal number of clusters is known, and clusters of each client are balanced in scale, as shown in (a1). By contrast, a real cluster may be incomplete on certain clients, and clients may contain clusters that comprise non-adjacent subclusters as indicated in (a2). As a result, existing FC approaches can be easily misled by the Non-ICD to form improper clusters at the server. As can be seen in (b), the clustering performance of existing state-of-the-art FC methods decreases sharply with the increase of Non-ICD degree.
• Figure 2: Overview of GOLD. Each client first employs FCPL to extract compact local subclusters from Non-ICD data, then transmits only the corresponding centroids to the server. The server executes MCPL, enabling the uploaded centroids to naturally aggregate across multiple granularity levels. The resulting multi-granular distributions are further encoded into an enhanced representation to drive the final feature-cluster importance learning-based clustering.
• Figure 3: Federated clustering performance comparison under different Non-ICD degrees ($\lambda$). A smaller $\lambda$ indicates lower data heterogeneity across clients.
• Figure 4: Accumulated granularity ablation of GOLD with progressively incorporated granularity levels. The left axis indicates the model performance at each accumulation stage, while the right axis shows the number of converged clusters corresponding to each granularity.
• Figure 5: Single-granular representation ablation of GOLD, evaluated by 4 indices. The dashed lines represent the performance of the complete GOLD, while the bars show the performance after ablating each granularity level.

Theorems & Definitions (9)

• Definition 1
• Remark 1
• Remark 2
• Remark 3
• Remark 4
• Lemma 1
• Lemma 2
• Lemma A.1
• Lemma A.2