Auto-Configuring Entity Resolution Pipelines

TL;DR

The paper tackles the automatic configuration of end-to-end entity resolution pipelines (ETEER) under two settings: with ground truth (Problem 1) and without ground truth (Problem 2). It first adapts hyperparameter optimization samplers (Random, TPES, QMCS, GPS) to select ER pipeline configurations, demonstrating that these methods approach grid-search performance while reducing search time by orders of magnitude. For the no-ground-truth scenario, it builds a regression framework using a rich set of dataset and configuration features, trained with Random Forest or AutoML, to predict and select effective configurations; Random Forest with grid-search instance generation provides the best balance of accuracy and efficiency and generalizes to unseen data. The work provides extensive empirical validation over 11 real-world datasets and releases its code openly, offering a practical pathway to automatic ER pipeline tuning in diverse settings.

Abstract

The same real-world entity (e.g., a movie, a restaurant, a person) may be described in various ways on different datasets. Entity Resolution (ER) aims to find such different descriptions of the same entity, this way improving data quality and, therefore, data value. However, an ER pipeline typically involves several steps (e.g., blocking, similarity estimation, clustering), with each step requiring its own configurations and tuning. The choice of the best configuration, among a vast number of possible combinations, is a dataset-specific and labor-intensive task both for novice and expert users, while it often requires some ground truth knowledge of real matches. In this work, we examine ways of automatically configuring a state of-the-art end-to-end ER pipeline based on pre-trained language models under two settings: (i) When ground truth is available. In this case, sampling strategies that are typically used for hyperparameter optimization can significantly restrict the search of the configuration space. We experimentally compare their relative effectiveness and time efficiency, applying them to ER pipelines for the first time. (ii) When no ground truth is available. In this case, labelled data extracted from other datasets with available ground truth can be used to train a regression model that predicts the relative effectiveness of parameter configurations. Experimenting with 11 ER benchmark datasets, we evaluate the relative performance of existing techniques that address each problem, but have not been applied to ER before.

Figures (8)

• Figure 1: The distribution of F1-scores for the end-to-end ER pipeline of Figure \ref{['fig:eeter_pipeline']}, when applying the 39,900 different configurations of Table \ref{['tab:parameter-values']} to the 10 real-world datasets of Table \ref{['tab:dataset-specs']}.
• Figure 2: The ETEER pipeline considered in this work.
• Figure 3: F1-score of the sampling-based search algorithms in Section \ref{['sec:problem-1']} as the number of trials increases from 5 to 100.
• Figure 4: The ratio between (a) the F1-score of sampling-based search algorithms and grid search, and (b) their run-times.
• Figure 5: Average F1 score per sampler and per trial, across all the datasets.

Theorems & Definitions (2)

• definition 1: End-to-end Entity Resolution (ETEER)
• definition 2: ETEER Pipeline