CGPT: Cluster-Guided Partial Tables with LLM-Generated Supervision for Table Retrieval

TL;DR

CGPT tackles the suboptimality of embedding entire tables by introducing semantically diverse partial tables created through K-means clustering, paired with LLM-generated synthetic queries and hard-negative contrastive fine-tuning. The framework combines clustering-based partial-table construction with an InfoNCE objective to refine embeddings, achieving robust cross-domain performance and strong improvements (average R@1 of $16.54\%$) across four public benchmarks. It demonstrates that semantically guided partial-table generation, coupled with supervision from LLM-generated queries, provides a scalable and effective paradigm for large-scale table retrieval, even with smaller LLMs. The approach shows strong cross-domain generalization and competitive results across varying LLMs, making it practical for real-world, multilingual, and multi-domain table retrieval tasks.

Abstract

General-purpose embedding models have demonstrated strong performance in text retrieval but remain suboptimal for table retrieval, where highly structured content leads to semantic compression and query-table mismatch. Recent LLM-based retrieval augmentation methods mitigate this issue by generating synthetic queries, yet they often rely on heuristic partial-table selection and seldom leverage these synthetic queries as supervision to improve the embedding model. We introduce CGPT, a training framework that enhances table retrieval through LLM-generated supervision. CGPT constructs semantically diverse partial tables by clustering table instances using K-means and sampling across clusters to broaden semantic coverage. An LLM then generates synthetic queries for these partial tables, which are used in hard-negative contrastive fine-tuning to refine the embedding model. Experiments across four public benchmarks (MimoTable, OTTQA, FetaQA, and E2E-WTQ) show that CGPT consistently outperforms retrieval baselines, including QGpT, with an average R@1 improvement of 16.54 percent. In a unified multi-domain corpus setting, CGPT further demonstrates strong cross-domain generalization and remains effective even when using smaller LLMs for synthetic query generation. These results indicate that semantically guided partial-table construction, combined with contrastive training from LLM-generated supervision, provides an effective and scalable paradigm for large-scale table retrieval. Our code is available at https://github.com/yumeow0122/CGPT.

Figures (1)

• Figure 1: Overall workflow of CGPT. The framework consists of four main stages: (1) clustering-based partial table generation; (2) synthetic query generation; (3) hard negative sampling; and (4) model fine-tuning.