ChatVis: Automating Scientific Visualization with a Large Language Model

TL;DR

An iterative assistant that can synthetically generate Python scripts for data analysis and visualization using a large language model (LLM) is developed, demonstrating correct execution on five canonical visualization scenarios and comparing results with ground truth.

Abstract

We develop an iterative assistant we call ChatVis that can synthetically generate Python scripts for data analysis and visualization using a large language model (LLM). The assistant allows a user to specify the operations in natural language, attempting to generate a Python script for the desired operations, prompting the LLM to revise the script as needed until it executes correctly. The iterations include an error detection and correction mechanism that extracts error messages from the execution of the script and subsequently prompts LLM to correct the error. Our method demonstrates correct execution on five canonical visualization scenarios, comparing results with ground truth. We also compared our results with scripts generated by several other LLMs without any assistance. In every instance, ChatVis successfully generated the correct script, whereas the unassisted LLMs failed to do so. The code is available on GitHub: https://github.com/tanwimallick/ChatVis/.

Figures (6)

• Figure 1: Logical flow of ChatVis execution. User input is processed by an LLM to create a specialized prompt, which, alongside example code snippets, helps the LLM generate a script. This script is executed and any error messages are fed back into the LLM for iterative improvement until a 2D visualization is successfully produced.
• Figure 2: Generated images for isosurfacing.
• Figure 3: Generated images for slicing followed by contouring.
• Figure 4: Generated images for volume rendering.
• Figure 5: Generated images for Delaunay triangulation.