Gem: Gaussian Mixture Model Embeddings for Numerical Feature Distributions

TL;DR

This paper proposes a method called Gem (Gaussian mixture model embeddings) that creates embeddings that build on numerical value distributions from columns using a Gaussian Mixture Model to identify and cluster columns with similar value distributions.

Abstract

Embeddings are now used to underpin a wide variety of data management tasks, including entity resolution, dataset search and semantic type detection. Such applications often involve datasets with numerical columns, but there has been more emphasis placed on the semantics of categorical data in embeddings than on the distinctive features of numerical data. In this paper, we propose a method called Gem (Gaussian mixture model embeddings) that creates embeddings that build on numerical value distributions from columns. The proposed method specializes a Gaussian Mixture Model (GMM) to identify and cluster columns with similar value distributions. We introduce a signature mechanism that generates a probability matrix for each column, indicating its likelihood of belonging to specific Gaussian components, which can be used for different applications, such as to determine semantic types. Finally, we generate embeddings for three numerical data properties: distributional, statistical, and contextual. Our core method focuses solely on numerical columns without using table names or neighboring columns for context. However, the method can be combined with other types of evidence, and we later integrate attribute names with the Gaussian embeddings to evaluate the method's contribution to improving overall performance. We compare Gem with several baseline methods for numeric only and numeric + context tasks, showing that Gem consistently outperforms the baselines on four benchmark datasets.

Figures (5)

• Figure 1: A histogram with a Kernel Density Estimate (KDE) overlay distributions of four numerical columns: Age, Rank, Test Score, and Temperature. Despite the similar distribution shapes — Age and Rank both showing a normal distribution around a mean of 30 and Test Score and Temperature around a mean of 75 — the semantic contexts differ significantly and refer to the different semantic types and units. For example, "Age" might be measured in years, "Rank" in a hierarchical position, "Test Score" as points out of 100, and "Temperature" in degrees, Fahrenheit and Celsius. These variations illustrate the complexity of semantic type detection of columns with different distributions. In this context, existing methods struggle to distinguish these overlapping columns. However, our proposal can effectively distinguish between these columns by focusing on their distributional properties.
• Figure 2: The process of transforming a table with three numeric columns (Price, Quantity, Discount) into a final embedding matrix. First, the GMM is fitted to the values in each column. For each value $x_n$ in a column, the probability $p(x_n \mid \mu_j, \Sigma_j)$ that it belongs to each component $C_j$ of the GMM is calculated using Equation \ref{['eq6']} where $\mu_j$ and $\Sigma_j$ are the mean and covariance matrix of component $j$, respectively. Next, the mean probabilities for each component are computed: $\mu_{C_j} = \frac{1}{N} \sum_{i=1}^{N} p(h_i \mid \mu_j, \Sigma_j)$ where $N$ is the number of values in the column. These mean probabilities are augmented with additional statistical features $(s1, s2, s3 ... sn)$. Simultaneously, the column headers are transformed into embeddings using the SBERT model. Finally, the normalized probability matrix (value embeddings) and the normalized SBERT embeddings (header embeddings) are combined to form the final embedding matrix for the table. The final embedding vector for each column includes the distributional embeddings using GMM, the statistical embeddings using data properties, and the contextual embeddings from headers, resulting in a comprehensive representation of the column data.
• Figure 3: Average Precision for WDC and GDS across different feature settings. 'D' represents distributional features, 'S' denotes statistical features and 'C' refers to contextual (headers) features. The results illustrate the performance of these feature combinations for both the WDC and GDS datasets.
• Figure 4: Performance comparison across different numbers of GMM components for all datasets.
• Figure 5: Run time comparison of different methods: (a) Overall view; (b) Zoomed-in view.