Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Evaluating Prompting Strategies for Chart Question Answering with Large Language Models

Ruthuparna Naikar, Ying Zhu

Abstract

Prompting strategies affect LLM reasoning performance, but their role in chart-based QA remains underexplored. We present a systematic evaluation of four widely used prompting paradigms (Zero-Shot, Few-Shot, Zero-Shot Chain-of-Thought, and Few-Shot Chain-of-Thought) across GPT-3.5, GPT-4, and GPT-4o on the ChartQA dataset. Our framework operates exclusively on structured chart data, isolating prompt structure as the only experimental variable, and evaluates performance using two metrics: Accuracy and Exact Match. Results from 1,200 diverse ChartQA samples show that Few-Shot Chain-of-Thought prompting consistently yields the highest accuracy (up to 78.2\%), particularly on reasoning-intensive questions, while Few-Shot prompting improves format adherence. Zero-Shot performs well only with high-capacity models on simpler tasks. These findings provide actionable guidance for selecting prompting strategies in structured data reasoning tasks, with implications for both efficiency and accuracy in real-world applications.

Evaluating Prompting Strategies for Chart Question Answering with Large Language Models

Abstract

Prompting strategies affect LLM reasoning performance, but their role in chart-based QA remains underexplored. We present a systematic evaluation of four widely used prompting paradigms (Zero-Shot, Few-Shot, Zero-Shot Chain-of-Thought, and Few-Shot Chain-of-Thought) across GPT-3.5, GPT-4, and GPT-4o on the ChartQA dataset. Our framework operates exclusively on structured chart data, isolating prompt structure as the only experimental variable, and evaluates performance using two metrics: Accuracy and Exact Match. Results from 1,200 diverse ChartQA samples show that Few-Shot Chain-of-Thought prompting consistently yields the highest accuracy (up to 78.2\%), particularly on reasoning-intensive questions, while Few-Shot prompting improves format adherence. Zero-Shot performs well only with high-capacity models on simpler tasks. These findings provide actionable guidance for selecting prompting strategies in structured data reasoning tasks, with implications for both efficiency and accuracy in real-world applications.
Paper Structure (22 sections, 3 figures, 5 tables)

This paper contains 22 sections, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. LLMs in Data Visualizations
  4. Prompt Strategies for Large Language Models
  5. Visual Question Answering Over Charts
  6. Methodology
  7. Large Language Models and Prompting
  8. Architecture Overview
  9. Data
  10. Model Selection
  11. Experimental Setup
  12. Prompting Strategies
  13. Prompt Construction
  14. Prompt Variants
  15. Prompt Examples
  16. ...and 7 more sections

Figures (3)

  • Figure 1: This diagram illustrates the workflow of chart-based question answering with LLMs. Structured chart data and natural language questions are transformed into prompts, which are submitted to an instruction-tuned LLM to generate answers.
  • Figure 2: Sample data visualization and its serialized JSON representation.
  • Figure 3: Generalized prompt template with components.