Development of Data Evaluation Benchmark for Data Wrangling Recommendation System
Yuqing Wang, Anna Fariha
TL;DR
The paper addresses the difficulty of providing context-aware data-wrangling recommendations across diverse real-world datasets. It introduces a benchmarking system that mines 10,000 Python notebooks from Kaggle (spanning roughly 1,000 datasets) and converts code into a structured JSON representation via Python AST, enabling systematic analysis of data-wrangling operations and variable usage. A reproducible pipeline collects notebooks and datasets through the Kaggle API, extracts operation types (e.g., as_type, datetime, apply, fillna) and variable roles (dictionaries, functions, lambdas), and outputs an Excel-based summary for benchmarking. Key contributions include a Python module for benchmark creation, a JSON representation of notebook transformations, and a quality-control framework to ensure representative data, offering insights into common wrangling patterns and informing improvements in recommendation systems. The work lays the groundwork for robust evaluation of data-wrangling recommendations and paves the way for expanding sources and refining guidance for practitioners.
Abstract
CoWrangler is a data-wrangling recommender system designed to streamline data processing tasks. Recognizing that data processing is often time-consuming and complex for novice users, we aim to simplify the decision-making process regarding the most effective subsequent data operation. By analyzing over 10,000 Kaggle notebooks spanning approximately 1,000 datasets, we derive insights into common data processing strategies employed by users across various tasks. This analysis helps us understand how dataset quality influences wrangling operations, informing our ongoing efforts to possibly expand our dataset sources in the future.