Toward Reliable Scientific Visualization Pipeline Construction with Structure-Aware Retrieval-Augmented LLMs

Abstract

Scientific visualization pipelines encode domain-specific procedural knowledge with strict execution dependencies, making their construction sensitive to missing stages, incorrect operator usage, or improper ordering. Thus, generating executable scientific visualization pipelines from natural-language descriptions remains challenging for large language models, particularly in web-based environments where visualization authoring relies on explicit code-level pipeline assembly. In this work, we investigate the reliability of LLM-based scientific visualization pipeline generation, focusing on vtk.js as a representative web-based visualization library. We propose a structure-aware retrieval-augmented generation workflow that provides pipeline-aligned vtk.js code examples as contextual guidance, supporting correct module selection, parameter configuration, and execution order. We evaluate the proposed workflow across multiple multi-stage scientific visualization tasks and LLMs, measuring reliability in terms of pipeline executability and human correction effort. To this end, we introduce correction cost as metric for the amount of manual intervention required to obtain a valid pipeline. Our results show that structured, domain-specific context substantially improves pipeline executability and reduces correction cost. We additionally provide an interactive analysis interface to support human-in-the-loop inspection and systematic evaluation of generated visualization pipelines.

Figures (8)

• Figure 1: (a) shows raw code generated by ChatGPT-4o, (b) shows the fixed code based on raw code. (c) shows results using fixed code as context information and let qwen3:8b to create a sphere source instead of cone. We only show vtk.js code.
• Figure 2: The workflow presented in this paper includes four key modules: code corpus preparation, RAG workflow, result evaluation and visualization system.
• Figure 3: Overview of the Corpus Preparation
• Figure 4: Details of the Structure-Aware Retrieval-Augmented Generation (RAG) workflow
• Figure 5: The interactive visualization system supports end-to-end construction and verification of scientific visualization pipelines. The interface is organized into three regions: the left panel (a–c) implements the multi-stage augmentation workflow, the middle panel (d) provides synchronized views for code editing and rendering inspection, and the right panel (e) supports multi-dimensional assessment. In particular, (a) Configuration region allows users to enter natural-language instructions and select target models. (b) Pipeline Planning region visualizes the decomposed, pipeline-aligned steps, enabling users to refine intermediate logic before generation. (c) Corpus Retrieval presents retrieved reference cases with functional summaries and visual previews, and includes a rejection mechanism to filter irrelevant context. (d) Rendering and Editing offers a dual-view workspace to debug generated code by comparing its rendered output with the expected result. (e) Evaluation Panel reports automated scores and enables human-in-the-loop qualitative assessment across multiple criteria.