Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SketchFill: Sketch-Guided Code Generation for Imputing Derived Missing Values

Yunfan Zhang, Changlun Li, Yuyu Luo, Nan Tang

TL;DR

SketchFill introduces a sketch-guided, self-reflective pipeline for derived missing value imputation (DMVI) that combines a Meta-Sketch with a Domain-Sketch to guide LLM-driven code generation. The framework integrates a Domain-Sketch Generator, Self-Reflected Code Generator, Summarizer, and Executor to produce executable Python functions for DMVI, iteratively refined by a Reflector. Empirical results on five domain datasets show SketchFill substantially outperforms CoT-, Code Generation-, and MetaGPT-based approaches, establishing a new standard for automated data cleaning. The work highlights the importance of explicit knowledge manifestation and iterative refinement in LLM-assisted DMVI, with practical impact for reliable numerical imputations across heterogeneous datasets.

Abstract

Missing value is a critical issue in data science, significantly impacting the reliability of analyses and predictions. Missing value imputation (MVI) is a longstanding problem because it highly relies on domain knowledge. Large language models (LLMs) have emerged as a promising tool for data cleaning, including MVI for tabular data, offering advanced capabilities for understanding and generating content. However, despite their promise, existing LLM techniques such as in-context learning and Chain-of-Thought (CoT) often fall short in guiding LLMs to perform complex reasoning for MVI, particularly when imputing derived missing values, which require mathematical formulas and data relationships across rows and columns. This gap underscores the need for further advancements in LLM methodologies to enhance their reasoning capabilities for more reliable imputation outcomes. To fill this gap, we propose SketchFill, a novel sketch-based method to guide LLMs in generating accurate formulas to impute missing numerical values. Our experimental results demonstrate that SketchFill significantly outperforms state-of-the-art methods, achieving 56.2% higher accuracy than CoT-based methods and 78.8% higher accuracy than MetaGPT. This sets a new standard for automated data cleaning and advances the field of MVI for numerical values.

SketchFill: Sketch-Guided Code Generation for Imputing Derived Missing Values

TL;DR

SketchFill introduces a sketch-guided, self-reflective pipeline for derived missing value imputation (DMVI) that combines a Meta-Sketch with a Domain-Sketch to guide LLM-driven code generation. The framework integrates a Domain-Sketch Generator, Self-Reflected Code Generator, Summarizer, and Executor to produce executable Python functions for DMVI, iteratively refined by a Reflector. Empirical results on five domain datasets show SketchFill substantially outperforms CoT-, Code Generation-, and MetaGPT-based approaches, establishing a new standard for automated data cleaning. The work highlights the importance of explicit knowledge manifestation and iterative refinement in LLM-assisted DMVI, with practical impact for reliable numerical imputations across heterogeneous datasets.

Abstract

Missing value is a critical issue in data science, significantly impacting the reliability of analyses and predictions. Missing value imputation (MVI) is a longstanding problem because it highly relies on domain knowledge. Large language models (LLMs) have emerged as a promising tool for data cleaning, including MVI for tabular data, offering advanced capabilities for understanding and generating content. However, despite their promise, existing LLM techniques such as in-context learning and Chain-of-Thought (CoT) often fall short in guiding LLMs to perform complex reasoning for MVI, particularly when imputing derived missing values, which require mathematical formulas and data relationships across rows and columns. This gap underscores the need for further advancements in LLM methodologies to enhance their reasoning capabilities for more reliable imputation outcomes. To fill this gap, we propose SketchFill, a novel sketch-based method to guide LLMs in generating accurate formulas to impute missing numerical values. Our experimental results demonstrate that SketchFill significantly outperforms state-of-the-art methods, achieving 56.2% higher accuracy than CoT-based methods and 78.8% higher accuracy than MetaGPT. This sets a new standard for automated data cleaning and advances the field of MVI for numerical values.
Paper Structure (32 sections, 13 equations, 5 figures, 2 tables, 1 algorithm)

This paper contains 32 sections, 13 equations, 5 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Rethink: How Do Data Scientists Perform DMVI?
  3. Our Proposal: Sketch-Guided Self-Reflected Code Generation.
  4. Related Work
  5. Methodology
  6. Preliminaries
  7. SketchFill Overview
  8. Domain-Sketch Generator
  9. Self-Reflected Code Generator
  10. Summarizer
  11. Executor
  12. Experiment
  13. Environment Setup
  14. Result Analysis
  15. Discussion on other approaches
  16. ...and 17 more sections

Figures (5)

  • Figure 1: DMVI samples from our experimental datasets. The NaN represents the missing values and the formula on the bottom is the derived solution for the missing value imputation.
  • Figure 2: Different LLM-based approaches for DMVI
  • Figure 3: The SketchFill framework
  • Figure 4: (a) SketchFill performance across 5 datasets showing imputation accuracy compared with different approaches. (b) SketchFill performance across 5 datasets showing imputation find accuracy compared with different approaches. LLMs are backend by GPT-4o.
  • Figure 5: Llama3 imputation accuracy