OmniFC: Rethinking Federated Clustering via Lossless and Secure Distance Reconstruction

TL;DR

OmniFC addresses privacy leakage and robustness under Non-IID data in federated clustering by reconstructing the global distance matrix $\boldsymbol{D}$ in a lossless, secure manner using Lagrange coded computing. The framework decouples from model-specific proxies and enables a unified, one-shot extension of centralized clustering methods to the federated setting, with theoretical guarantees for exact distance reconstruction and $t$-private security when $m \ge 2l + 2t - 1$. Empirically, OmniFC achieves near-centralized performance for distance-dependent methods like spectral clustering and robust performance for model-based approaches (KM, FCM, NMF) across diverse datasets and Non-IID scenarios, while demonstrating generality to additional clustering paradigms (KMed, DBSCAN, HC). The results suggest a practical pathway to privacy-preserving, scalable federated clustering and hint at future use in federated graph learning and related domains. Code will be released.

Abstract

Federated clustering (FC) aims to discover global cluster structures across decentralized clients without sharing raw data, making privacy preservation a fundamental requirement. There are two critical challenges: (1) privacy leakage during collaboration, and (2) robustness degradation due to aggregation of proxy information from non-independent and identically distributed (Non-IID) local data, leading to inaccurate or inconsistent global clustering. Existing solutions typically rely on model-specific local proxies, which are sensitive to data heterogeneity and inherit inductive biases from their centralized counterparts, thus limiting robustness and generality. We propose Omni Federated Clustering (OmniFC), a unified and model-agnostic framework. Leveraging Lagrange coded computing, our method enables clients to share only encoded data, allowing exact reconstruction of the global distance matrix--a fundamental representation of sample relationships--without leaking private information, even under client collusion. This construction is naturally resilient to Non-IID data distributions. This approach decouples FC from model-specific proxies, providing a unified extension mechanism applicable to diverse centralized clustering methods. Theoretical analysis confirms both reconstruction fidelity and privacy guarantees, while comprehensive experiments demonstrate OmniFC's superior robustness, effectiveness, and generality across various benchmarks compared to state-of-the-art methods. Code will be released.

Figures (6)

• Figure 1: Robustness to heterogeneity. We employ COIL-100 and 100 clients to compare the proposed OmniFC with the federated extensions of centralized clustering methods dennis2021heterogeneitypan2023machinestallmann2022towardsqiao2023federatedwang2022federated. Compared to existing one-to-one extensions, OmniFC not only unifies the extension of centralized clustering methods but also achieves superior robustness and effectiveness.
• Figure 2: An overview of the proposed OmniFC. The architecture comprises three main steps: 1) Local Lagrange-Encoded Sharing. Each client $j$$(j \in [m])$ encodes its private data using Lagrange polynomial interpolation and distributes the encoded data to all peers, enabling each client to construct a global encoded dataset while preserving data privacy. 2) Global Distance Reconstruction. Each client $j$ computes pairwise distances within its global encoded dataset and transmits the results to the central server, which leverages them to reconstruct the global distance matrix. 3) Cluster Assignment. A centralized clustering method (e.g., k-means) is applied to the global distance matrix to produce the final clustering result $\pi^*$.
• Figure 3: An illustration of the Lagrange encoding. Each sample $\boldsymbol{\tilde{x}}_i$$(i \in [n])$ is initially divided into $l$ segments $\{\boldsymbol{s}_{i, o}\}_{o = 1}^{l}$. Incorporating $t$ additional noises $\{\boldsymbol{s}_{i, l+o}\}_{o = 1}^{t}$, Lagrange interpolation is then conducted as per Equation (\ref{['get_f1']}) to yield $\boldsymbol{f}_{\boldsymbol{\tilde{x}}_i}(\alpha)$. Subsequently, the encoded representations $\{\boldsymbol{z}_{i, j} \}_{j = 1}^{m}$ of $\boldsymbol{\tilde{x}}_i$ are computed according to Equation (\ref{['get_z']}).
• Figure 4: Comparison between the ground-truth (top row) and reconstructed (bottom row) pairwise distance matrices. The visual consistency indicates that the proposed OmniFC faithfully recovers the inter-sample similarity.
• Figure 5: Hyperparameter sensitivity of the global distance matrix reconstruction loss. The gray-highlighted region denotes hyperparameter settings that violate the condition $m \geq 2l + 2t - 1$ in Theorem \ref{['the1']}, thus precluding distance reconstruction.

Theorems & Definitions (9)

• Theorem 1
• Remark 1
• Theorem 2
• Remark 2
• Remark 3
• Lemma 1
• Lemma 2
• proof
• proof