Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SEED: Domain-Specific Data Curation With Large Language Models

Zui Chen, Lei Cao, Sam Madden, Tim Kraska, Zeyuan Shang, Ju Fan, Nan Tang, Zihui Gu, Chunwei Liu, Michael Cafarella

TL;DR

SEED introduces an LLM-as-compiler framework that automatically builds instance-optimized data curation pipelines from modular operators (CodeGen, CacheReuse, ModelGen, LLM). A skyline-based SEED optimizer selects module configurations and ordering to minimize execution cost while maintaining strong accuracy, with dynamic re-optimization as data and caches evolve. The system integrates infrastructure for caching, batching, and tool use, and includes CodeGen ensembles and small-model distillation to reduce reliance on expensive LLM calls. Empirical results across 9 datasets and 5 tasks show state-of-the-art or competitive few-shot performance with substantially fewer LLM invocations, validating SEED’s practical impact for domain-specific data curation. The work demonstrates a principled, reusable approach to constructing data curation solutions that adapt to task, dataset, and domain requirements without manual coding or heavy prompt engineering.

Abstract

Data curation tasks that prepare data for analytics are critical for turning data into actionable insights. However, due to the diverse requirements of applications in different domains, generic off-the-shelf tools are typically insufficient. As a result, data scientists often have to develop domain-specific solutions tailored to both the dataset and the task, e.g. writing domain-specific code or training machine learning models on a sufficient number of annotated examples. This process is notoriously difficult and time-consuming. We present SEED, an LLM-as-compiler approach that automatically generates domain-specific data curation solutions via Large Language Models (LLMs). Once the user describes a task, input data, and expected output, the SEED compiler produces a hybrid pipeline that combines LLM querying with more cost-effective alternatives, such as vector-based caching, LLM-generated code, and small models trained on LLM-annotated data. SEED features an optimizer that automatically selects from the four LLM-assisted modules and forms a hybrid execution pipeline that best fits the task at hand. To validate this new, revolutionary approach, we conducted experiments on $9$ datasets spanning over $5$ data curation tasks. In comparison to solutions that use the LLM on every data record, SEED achieves state-of-the-art or comparable few-shot performance, while significantly reducing the number of LLM calls.

SEED: Domain-Specific Data Curation With Large Language Models

TL;DR

SEED introduces an LLM-as-compiler framework that automatically builds instance-optimized data curation pipelines from modular operators (CodeGen, CacheReuse, ModelGen, LLM). A skyline-based SEED optimizer selects module configurations and ordering to minimize execution cost while maintaining strong accuracy, with dynamic re-optimization as data and caches evolve. The system integrates infrastructure for caching, batching, and tool use, and includes CodeGen ensembles and small-model distillation to reduce reliance on expensive LLM calls. Empirical results across 9 datasets and 5 tasks show state-of-the-art or competitive few-shot performance with substantially fewer LLM invocations, validating SEED’s practical impact for domain-specific data curation. The work demonstrates a principled, reusable approach to constructing data curation solutions that adapt to task, dataset, and domain requirements without manual coding or heavy prompt engineering.

Abstract

Data curation tasks that prepare data for analytics are critical for turning data into actionable insights. However, due to the diverse requirements of applications in different domains, generic off-the-shelf tools are typically insufficient. As a result, data scientists often have to develop domain-specific solutions tailored to both the dataset and the task, e.g. writing domain-specific code or training machine learning models on a sufficient number of annotated examples. This process is notoriously difficult and time-consuming. We present SEED, an LLM-as-compiler approach that automatically generates domain-specific data curation solutions via Large Language Models (LLMs). Once the user describes a task, input data, and expected output, the SEED compiler produces a hybrid pipeline that combines LLM querying with more cost-effective alternatives, such as vector-based caching, LLM-generated code, and small models trained on LLM-annotated data. SEED features an optimizer that automatically selects from the four LLM-assisted modules and forms a hybrid execution pipeline that best fits the task at hand. To validate this new, revolutionary approach, we conducted experiments on datasets spanning over data curation tasks. In comparison to solutions that use the LLM on every data record, SEED achieves state-of-the-art or comparable few-shot performance, while significantly reducing the number of LLM calls.
Paper Structure (35 sections, 2 theorems, 2 equations, 4 figures, 11 tables, 3 algorithms)

This paper contains 35 sections, 2 theorems, 2 equations, 4 figures, 11 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Our Approach: SEED
  3. Contributions
  4. SEED Overview
  5. SEED Usage
  6. SEED internals
  7. SEED Optimizer
  8. Generic SEED Optimizer
  9. Specialized SEED Optimizer
  10. Dynamic Optimization
  11. SEED Modules
  12. CodeGen Module
  13. ModelGen Module
  14. Infrastructure
  15. Cache Storage
  16. ...and 20 more sections

Key Result

Theorem 3.1

As long as the descendants of a dominated subplan are also dominated, the SEED optimizer is guaranteed to find the optimal solution $\mathcal{P}^\ast = \max_{\mathcal{P}}\ \mathcal{A}(\mathcal{P})$ as well as the most efficient solution $\min_{\mathcal{P}}\ \mathcal{C}(\mathcal{P})$ subject to: $\ma

Figures (4)

  • Figure 1: The Architecture of SEED, using an entity resolution task as example.
  • Figure 2: Illustration of the SEED Optimizer.
  • Figure 3: Illustration of one evolution iteration.
  • Figure 4: Illustration of iterative tool invocation.

Theorems & Definitions (5)

  • Definition 3.1: SEED Optimization
  • Definition 3.2: Skyline Plan
  • Theorem 3.1
  • Definition 3.3: The Execution Cost of SEED Plan
  • Theorem 3.2