ScaleViz: Scaling Visualization Recommendation Models on Large Data

TL;DR

ScaleViz tackles the scalability bottleneck in automated visualization recommendation by introducing a budget-aware reinforcement-learning pipeline. It combines a cost profiler that extrapolates per-feature computation costs with a double deep Q-learning agent that sequentially selects a subset of statistics under a user-defined time budget, preserving model performance. Evaluations on VizML and MLVR across four large datasets show up to $\approx 10\times$ speedups with minimal degradation in accuracy, and analyses reveal strong dataset-specific feature selection and favorable scaling behavior as data size grows. The approach enables practical, cost-effective Vis-Rec on large-scale datasets, aligning visualization quality with real-world time constraints.

Abstract

Automated visualization recommendations (vis-rec) help users to derive crucial insights from new datasets. Typically, such automated vis-rec models first calculate a large number of statistics from the datasets and then use machine-learning models to score or classify multiple visualizations choices to recommend the most effective ones, as per the statistics. However, state-of-the art models rely on very large number of expensive statistics and therefore using such models on large datasets become infeasible due to prohibitively large computational time, limiting the effectiveness of such techniques to most real world complex and large datasets. In this paper, we propose a novel reinforcement-learning (RL) based framework that takes a given vis-rec model and a time-budget from the user and identifies the best set of input statistics that would be most effective while generating the visual insights within a given time budget, using the given model. Using two state-of-the-art vis-rec models applied on three large real-world datasets, we show the effectiveness of our technique in significantly reducing time-to visualize with very small amount of introduced error. Our approach is about 10X times faster compared to the baseline approaches that introduce similar amounts of error.

Figures (4)

• Figure 1: CDF of computation time (in seconds) of various features on datasets of different sizes. With increase in dataset size, the computation time of complex features drastically increases.
• Figure 2: Pipeline overview: The cost profiler estimates the computational cost for computing statistics. RL agent training begins with a set of zero-cost features. Within each episode, the agent dynamically acquires features until the budget is exhausted. The $Q$-value is estimated using rewards which is based on increased certainty in recommendations with newly acquired features, considering acquisition costs. This iterative process continues until error converges below a certain error value. Once trained, in the inference pipeline, the RL agent now selects features for the specified budget, tailored to the dataset and model.
• Figure 3: Evaluation of VizML on different datasets
• Figure 4: Evaluation of MLVR on different datasets