Consistent data fusion with Parker

TL;DR

It is shown that the well-known set cover methodology can be adapted to the setting of EPKs and this yields an efficient algorithm to find minimal cost repairs of sources, implemented in a repair engine called Parker.

Abstract

When combining data from multiple sources, inconsistent data complicates the production of a coherent result. In this paper, we introduce a new type of constraints called edit rules under a partial key (EPKs). These constraints can model inconsistencies both within and between sources, but in a loosely-coupled matter. We show that we can adapt the well-known set cover methodology to the setting of EPKs and this yields an efficient algorithm to find minimal cost repairs of sources. This algorithm is implemented in a repair engine called Parker. Empirical results show that Parker is several orders of magnitude faster than state-of-the-art repair tools. At the same time, the quality of the repairs in terms of $F_1$-score ranges from comparable to better compared to these tools.

Figures (5)

• Figure 1: Data on the design parameters of a clinical trial executed in different sites (a). Edit rules that need to hold are shown in (b) and violations are marked in bold red font. Required agreement between sites is modelled by a partial key (c), with violations marked in grey. A minimal repair is shown in (d), with changes marked in bold green font.
• Figure 2: High-level architecture of the Parker repair engine.
• Figure 3: A modification of the data in Figure \ref{['fig:running-example']}
• Figure 4: Frequency of errors in the different attributes across the datasets.
• Figure 5: Precision and recall per attribute of repairs of the studied approaches. Attributes are ordered by the frequency of errors as of Figure \ref{['fig:errors_datasets']}.

Theorems & Definitions (16)

• Definition 1
• Definition 2
• Definition 3
• Proposition 1
• Definition 4
• Proposition 2
• Theorem 1
• Proposition 3
• Proposition 4
• Theorem 2