nvBench 2.0: Resolving Ambiguity in Text-to-Visualization through Stepwise Reasoning

TL;DR

nvBench 2.0 introduces the first ambiguity-aware benchmark for Text-to-Visualization, enabling evaluation of how systems handle underspecified natural language queries. Using a controlled ambiguity-injection pipeline, it provides 7,878 NL queries, 24,076 visualizations, and traceable stepwise reasoning paths across 780 tables in 153 domains. The authors also propose Step-Text2Vis, an LLM-based model trained with stepwise preference optimization (Step-DPO) that leverages the dataset's intelligible reasoning paths, achieving state-of-the-art F1@3 and F1@5 on ambiguous Text2VIS tasks. The work demonstrates that decomposing visualization reasoning into explicit steps improves accuracy and interpretability, offering a practical framework for building more robust Text2VIS systems in real-world, ambiguous-query scenarios.

Abstract

Text-to-Visualization (Text2VIS) enables users to create visualizations from natural language queries, making data insights more accessible. However, Text2VIS faces challenges in interpreting ambiguous queries, as users often express their visualization needs in imprecise language. To address this challenge, we introduce nBench 2.0, a new benchmark designed to evaluate Text2VIS systems in scenarios involving ambiguous queries. nvBench 2.0 includes 7,878 natural language queries and 24,076 corresponding visualizations, derived from 780 tables across 153 domains. It is built using a controlled ambiguity-injection pipeline that generates ambiguous queries through a reverse-generation workflow. By starting with unambiguous seed visualizations and selectively injecting ambiguities, the pipeline yields multiple valid interpretations for each query, with each ambiguous query traceable to its corresponding visualization through step-wise reasoning paths. We evaluate various Large Language Models (LLMs) on their ability to perform ambiguous Text2VIS tasks using nBench 2.0. We also propose Step-Text2Vis, an LLM-based model trained on nvBench 2.0, which enhances performance in ambiguous scenarios through step-wise preference optimization. Our results show that Step-Text2Vis outperforms all baselines, setting a new state-of-the-art for ambiguous Text2VIS tasks. Our source code and data are available at https://nvbench2.github.io/

Figures (10)

• Figure 1: Example of reasoning appropriate visualizations from an ambiguous query.
• Figure 2: The Pipeline for Synthesizing nvBench 2.0.
• Figure 3: Key Statistics of nvBench 2.0.
• Figure 4: F1 scores across different models and ambiguity levels. The figure is organized as a 4$\times$4 grid where columns represent increasing ambiguity levels, and rows represent different model groups. The first two rows display GPT, Claude model families with prompting-based methods. The last two rows display Qwen model families of prompting-based, supervised and preference learning methods, including our proposed Step-Text2Vis in the bottom row. Each radar chart displays F1@5 scores across six chart types, where larger polygons indicate better performance.
• Figure 5: Recall@5 across different models and ambiguity levels. The blue dashed horizontal line indicates the performance of our proposed Step-Text2Vis method, while the grey dashed horizontal line represents Qwen2.5-7B-SFT, which serves as the base model for our approach.