Prompt4Vis: Prompting Large Language Models with Example Mining and Schema Filtering for Tabular Data Visualization

TL;DR

Prompt4Vis tackles the challenge of generating data visualizations from natural language by leveraging large language models with in-context learning. It introduces two novelties—multi-objective example mining and schema filtering—to select effective demonstrations and prune database schemas, respectively. Through NVBench experiments, Prompt4Vis achieves notable improvements over prior baselines, demonstrating strong cross-domain stability and marking the first integration of in-context learning into text-to-vis. The work suggests that carefully curated prompts and streamlined schemas can significantly enhance LLM-based DVQ generation, paving the way for more generalizable NL-driven visualization systems.

Abstract

Data visualization (DV) systems are increasingly recognized for their profound capability to uncover insights from vast datasets, gaining attention across both industry and academia. Crafting data queries is an essential process within certain declarative visualization languages (DVLs, e.g., Vega-Lite, EChart.). The evolution of natural language processing (NLP) technologies has streamlined the use of natural language interfaces to visualize tabular data, offering a more accessible and intuitive user experience. However, current methods for converting natural language questions into data visualization queries, such as Seq2Vis, ncNet, and RGVisNet, despite utilizing complex neural network architectures, still fall short of expectations and have great room for improvement. Large language models (LLMs) such as ChatGPT and GPT-4, have established new benchmarks in a variety of NLP tasks, fundamentally altering the landscape of the field. Inspired by these advancements, we introduce a novel framework, Prompt4Vis, leveraging LLMs and in-context learning to enhance the performance of generating data visualization from natural language. Prompt4Vis comprises two key components: (1) a multi-objective example mining module, designed to find out the truly effective examples that strengthen the LLM's in-context learning capabilities for text-to-vis; (2) a schema filtering module, which is proposed to simplify the schema of the database. Extensive experiments through 5-fold cross-validation on the NVBench dataset demonstrate the superiority of Prompt4Vis, which notably surpasses the state-of-the-art (SOTA) RGVisNet by approximately 35.9% and 71.3% on dev and test sets, respectively. To the best of our knowledge, Prompt4Vis is the first work that introduces in-context learning into the text-to-vis for generating data visualization queries.

Figures (5)

• Figure 1: Pipeline of tabular data visualization.
• Figure 2: Workflow of Prompt4Vis, which prompts LLMs with an example mining module and a schema filtering module. The first module finds the truly effective examples for text-to-vis, and the second one simplifies the database schema.
• Figure 3: Training data construction for the influence model, which first finds similar questions with LLMs and takes each one of them as a prompt example. Then, these questions are sorted based on the influence scores of the DVQs generated by the LLMs. Finally, the positive and negative sets are established based on influence scores.
• Figure 4: Performance range in 5-fold cross-validation across various models and metrics.
• Figure 5: Overall accuracy with the different number of in-context examples.