Towards Scalable Visual Data Wrangling via Direct Manipulation

TL;DR

The paper tackles scalable visual data wrangling by introducing Buckaroo, a system that combines group-based anomaly detection, interactive visualization, and repair within a direct manipulation interface. It integrates a Postgres backend, localized anomaly detection with a group overlap graph, and two navigation modes (single-layer sampling and multi-layer pan-and-zoom via Hopara) to handle large datasets. Key contributions include the group abstraction for subgroup anomalies, an extensible detectors/wrappers API, efficient incremental updates, and automatic Python script generation for reproducibility, all demonstrated with expert feedback and runtime evaluations. The work aims to make data wrangling more accessible and reproducible for real-world large-scale datasets, with ongoing improvements for scalability and usability in practice.

Abstract

Data wrangling, the process of cleaning, transforming, and preparing data for analysis, is a well-known bottleneck in data science workflows. A wide range of data wrangling techniques have been proposed to mitigate this challenge. Of particular interest are visual data wrangling tools, in which users prepare data via graphical interactions (such as with visualizations) rather than requiring them to write scripts. We develop a visual data wrangling system, Buckaroo, that expands upon this paradigm by enabling the automatic discovery of interesting groups (e.g., Salary values for Country="Buthan") and identification of anomalies (e.g., missing values, outliers, and type mismatches) both within and across these groups. Crucially, this allows users to reason about how repairs applied to one group affect other groups in the dataset. A central challenge in visual data wrangling is scalability. Rendering entire datasets is often infeasible, yet showing only a small sample risks hiding rare but critical errors across groups. We address these challenges through carefully designed sampling strategies that prioritize errors, as well as novel aggregation techniques that support pan-and-zoom interactions over large datasets. Buckaroo maintains efficient indexing data structures and differential storage to localize anomaly detection and minimize recomputation. We demonstrate the applicability of our approach via an integration with the Hopara pan-and-zoom engine (enabling multi-layered navigation over large datasets without sacrificing interactivity). Finally, we explore our system's usability (via an expert review) and its scalability, finding that this design seems well matched with the challenges of this domain.

Figures (3)

• Figure 1: An overview of repairing an error through Buckaroo's user interface. highlights a user working through iterative and backtrack-laden process of cleaning a dataset. shows the full interface for a sample of the StackOverflow dataset. Each error type has a distinct color (e.g., red groups correspond to average anomalies), Upon selecting the group, Buckaroo shows a list of wrangling/repair actions on the right and shows a visual preview of the chart after the repair.
• Figure 2: Buckaroo system architecture. The user interface visualizes anomalies and supports interactive exploration, ranking errors, undo/redo, and script generation. The data selection and aggregation layer provides single-layer and multi-layer navigation through error sampling, distance-based sampling, drill-down, and panning. The backend implements built-in and custom error detectors. Buckaroo offers built-in and user-defined wrangling functions. The storage layer manages group generation, snapshot storage for undo/redo, error–tuple mappings, and indexes using PostgreSQL and AWS infrastructure.
• Figure 3: An overview of repairing an error through Buckaroo's user interface. Each error type has a distinct color (e.g., red groups correspond to average anomalies), Upon selecting the group , Buckaroo shows a list of wrangling/repair actions on the right and shows a visual preview of the chart after the repair.