An Efficient Proximity Graph-based Approach to Table Union Search
Yiming Xie, Hua Dai, Mingfeng Jiang, Pengyue Li, zhengkai Zhang, Bohan Li
TL;DR
This paper tackles Table Union Search in data lakes, where each table is encoded as a vector set and unionability requires a bipartite column assignment, making exact matching expensive. It introduces PGTUS, a proximity-graph-based, multi-stage framework that replaces costly one-to-one matching with many-to-one t-matching over partitioned vector sets, complemented by centroid-space refinement and an enhanced pruning strategy (PGTUS+). The approach leverages clustering-based matching and vector quantization to compress the vector-set repository, uses maximum weight many-to-one t-matching (MWMTO) to score candidates efficiently, and employs dual-ended priority queues to prune unpromising options. Empirical results across six data-lake benchmarks show 3.6–6.0× speedups with recall comparable to baselines, demonstrating scalability to web-scale data lakes while maintaining semantic alignment quality.
Abstract
Neural embedding models are extensively employed in the table union search problem, which aims to find semantically compatible tables that can be merged with a given query table. In particular, multi-vector models, which represent a table as a vector set (typically one vector per column), have been demonstrated to achieve superior retrieval quality by capturing fine-grained semantic alignments. However, this problem faces more severe efficiency challenges than the single-vector problem due to the inherent dependency on bipartite graph maximum matching to compute unionability scores. Therefore, this paper proposes an efficient Proximity Graph-based Table Union Search (PGTUS) approach. PGTUS employs a multi-stage pipeline that combines a novel refinement strategy, a filtering strategy based on many-to-one bipartite matching. Besides, we propose an enhanced pruning strategy to prune the candidate set, which further improve the search efficiency. Extensive experiments on six benchmark datasets demonstrate that our approach achieves 3.6-6.0X speedup over existing approaches while maintaining comparable recall rates.