Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Distributed Clustering based on Distributional Kernel

Hang Zhang, Yang Xu, Lei Gong, Ye Zhu, Kai Ming Ting

Abstract

This paper introduces a new framework for clustering in a distributed network called Distributed Clustering based on Distributional Kernel (K) or KDC that produces the final clusters based on the similarity with respect to the distributions of initial clusters, as measured by K. It is the only framework that satisfies all three of the following properties. First, KDC guarantees that the combined clustering outcome from all sites is equivalent to the clustering outcome of its centralized counterpart from the combined dataset from all sites. Second, the maximum runtime cost of any site in distributed mode is smaller than the runtime cost in centralized mode. Third, it is designed to discover clusters of arbitrary shapes, sizes and densities. To the best of our knowledge, this is the first distributed clustering framework that employs a distributional kernel. The distribution-based clustering leads directly to significantly better clustering outcomes than existing methods of distributed clustering. In addition, we introduce a new clustering algorithm called Kernel Bounded Cluster Cores, which is the best clustering algorithm applied to KDC among existing clustering algorithms. We also show that KDC is a generic framework that enables a quadratic time clustering algorithm to deal with large datasets that would otherwise be impossible.

Distributed Clustering based on Distributional Kernel

Abstract

This paper introduces a new framework for clustering in a distributed network called Distributed Clustering based on Distributional Kernel (K) or KDC that produces the final clusters based on the similarity with respect to the distributions of initial clusters, as measured by K. It is the only framework that satisfies all three of the following properties. First, KDC guarantees that the combined clustering outcome from all sites is equivalent to the clustering outcome of its centralized counterpart from the combined dataset from all sites. Second, the maximum runtime cost of any site in distributed mode is smaller than the runtime cost in centralized mode. Third, it is designed to discover clusters of arbitrary shapes, sizes and densities. To the best of our knowledge, this is the first distributed clustering framework that employs a distributional kernel. The distribution-based clustering leads directly to significantly better clustering outcomes than existing methods of distributed clustering. In addition, we introduce a new clustering algorithm called Kernel Bounded Cluster Cores, which is the best clustering algorithm applied to KDC among existing clustering algorithms. We also show that KDC is a generic framework that enables a quadratic time clustering algorithm to deal with large datasets that would otherwise be impossible.
Paper Structure (24 sections, 1 theorem, 3 equations, 10 figures, 6 tables, 2 algorithms)

This paper contains 24 sections, 1 theorem, 3 equations, 10 figures, 6 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. First Existing Approach: Framework $\dddot{\mathbbm{B}}$
  4. Second Existing Approach
  5. Distributional Kernel-based Clustering
  6. Clustering based on Distributional Kernel $\mathcal{K}$
  7. Properties of $\mathcal{K}$DC
  8. Distributed-Centralized Clustering Equivalence
  9. Communication Cost and Time Complexity
  10. Clustering Performance Guarantee
  11. Proposed Kernel-Bounded Cluster Cores
  12. Determinants of $\mathcal{K}$DC Property (c)
  13. A Comparison of $\kappa$BCC and SOTA Algorithm
  14. Experimental Designs and Settings
  15. Distributed Clustering Evaluation
  16. ...and 9 more sections

Key Result

Proposition 1

Assume that every point $\mathbf{x}$ in the given dataset $D$ belongs to only one of $k$ ground truth clusters $\mathcal{T}_i$, and they are non-overlapping clusters, i.e., $\mathcal{T}_i \cap \mathcal{T}_j = \emptyset\ \forall i \ne j$. The clusters $C_i$ produced from $D$ using the initial cluster $\hbox{has} \mathcal{P}_{\mathcal{C}_i} \approx \mathcal{P}_{\mathcal{T}_i} \hbox{if} \mathcal{P}_{

Figures (10)

  • Figure 1: A top-level comparison of distributed clustering: proposed Framework $\dddot{\mathbbm{A}}$ versus existing Framework $\dddot{\mathbbm{B}}$.
  • Figure 2: Clusters in the input space $\mathbb{R}^d$ (left), and cluster representations in the feature space $\mathbb{H}$ of distributional kernel $\mathcal{K}$ (right). Clusters $C$ as distributions $\mathcal{P}_C$ in $\mathbb{R}^d$, and as $\Phi(\mathcal{P}_C)$ in $\mathbb{H}$. Initial clusters $\mathcal{G} \subset C$ are identified in Framework $\mathbbm{A}$, given in Section \ref{['sec-FrameworkA']}.
  • Figure 3: The impact of using initial clusters $\mathcal{G}_i$ having $\mathcal{P}_{\mathcal{G}_i} \approx \mathcal{P}_{\mathcal{T}_i}$ in step 2; and compare distribution-based point assignment $\mathcal{K}(\delta(\mathbf{x}),\mathcal{P}_{\mathcal{G}_i})$ with center-based point assignment $\Vert \mathbf{x} - \bar{\mathbf{x}}_{\mathcal{G}_i} \Vert$ in step 3. Either the proposed Framework ${\mathbbm{A}}$ or ${\dddot{\mathbbm{A}}}$ produces the same clustering outcomes. NMI: normalized mutual information NMI2010
  • Figure 4: The clustering outcomes of $\mathcal{K}$DC or $\dddot{\mathbbm{A}}$ in steps 2 & 3 (and also in terms of NMI) using $k$-means, DP and $\kappa$BCC in step 2 on two datasets: Jain (top) and Complex9 (bottom). The final clustering outcomes are shown in the 'Step 3' row ($s=0.3n$ is used).
  • Figure 5: The comparison of ${\dddot{\mathbbm{A}}}$-$\kappa$BCC, ${\dddot{\mathbbm{A}}}$-$\kappa k$m (kernel $k$-means), coreset $k$-means in the existing framework and its kernel version (${\dddot{\mathbbm{B}}}$-$k$m and ${\dddot{\mathbbm{B}}}$-$\kappa k$m) in terms of NMI.
  • ...and 5 more figures

Theorems & Definitions (4)

  • Definition 1
  • Definition 2
  • Proposition 1
  • Definition 3