TableDC: Deep Clustering for Tabular Data

TL;DR

This paper presents a deep clustering algorithm for tabular data (TableDC) that reflects the properties of data management applications that cluster tables, rows, rows and columns and integrates Mahalanobis distance, offering a similarity method suitable for tabular data in high-dimensional latent spaces.

Abstract

Deep clustering (DC), a fusion of deep representation learning and clustering, has recently demonstrated positive results in data science, particularly text processing and computer vision. However, joint optimization of feature learning and data distribution in the multi-dimensional space is domain-specific, so existing DC methods struggle to generalize to other application domains (such as data integration and cleaning). In data management tasks, where high-density embeddings and overlapping clusters dominate, a data management-specific DC algorithm should be able to interact better with the data properties for supporting data cleaning and integration tasks. This paper presents a deep clustering algorithm for tabular data (TableDC) that reflects the properties of data management applications, particularly schema inference, entity resolution, and domain discovery. To address overlapping clusters, TableDC integrates Mahalanobis distance, which considers variance and correlation within the data, offering a similarity method suitable for tables, rows, or columns in high-dimensional latent spaces. TableDC provides flexibility for the final clustering assignment and shows higher tolerance to outliers through its heavy-tailed Cauchy distribution as the similarity kernel. The proposed similarity measure is particularly beneficial where the embeddings of raw data are densely packed and exhibit high degrees of overlap. Data cleaning tasks may involve a large number of clusters, which affects the scalability of existing DC methods. TableDC's self-supervised module efficiently learns data embeddings with a large number of clusters compared to existing benchmarks, which scale in quadratic time. We evaluated TableDC with several existing DC, Standard Clustering (SC), and state-of-the-art bespoke methods over benchmark datasets. TableDC consistently outperforms existing DC, SC, and bespoke methods.

Figures (5)

• Figure 1: Overview of TableDC framework. An autoencoder takes an embedding matrix as input, representing a set of rows for entity resolution, a set of columns for domain discovery, or a set of tables for schema inference. $\textit{z}$ is the latent representation obtained from the autoencoder, and $\textit{c}$ is initialized using Birch. In the self-supervised module, the $\textit{m}$ distribution is calculated from $\textit{q}$ by taking the Mahalanobis distance between $\textit{z}$ and $\textit{c}$ and setting the Cauchy distribution as a kernel to measure the similarity. $\textit{p}$ is the target distribution, which is also calculated from $\textit{q}$.
• Figure 2: TableDC vs. bespoke solutions for each problem. TableDC is integrated with SBERT in (a) and (b) and T5 in (c). In (b), Jaccard and Cosine are different similarity metrics of the JedAI framework DBLP:journals/is/PapadakisMGSTGB20.
• Figure 3: Scalability comparison of TableDC with existing DC approaches with respect to the number of clusters $\mathbb{K}$. DFCN and DCRN are not included in the comparison because we have not managed to run both on the available hardware resources with a high number of clusters.
• Figure 4: Impact of different cluster initialization methods on clustering performance. We used SBERT on web tables for schema inference, EmbDi on GeoSet for entity resolution, and SBERT on Camera for Domain Discovery.
• Figure 5: $re_{loss}$ and $\text{KL}_\text{div}$ (between $\textit{q}$ and $\textit{p}$) comparative analysis of TableDC with benchmark methods for schema inference on web tables data.