Do You See What I See? A Qualitative Study Eliciting High-Level Visualization Comprehension
Ghulam Jilani Quadri, Arran Zeyu Wang, Zhehao Wang, Jennifer Adorno, Paul Rosen, Danielle Albers Szafir
TL;DR
This paper investigates high-level visualization comprehension—how audiences intuitively interpret graphics without task framing—across line, bar, and scatter plots. Using a qualitative approach with think-aloud and written responses, it analyzes 60 stimuli derived from real-world designs and codes 288 participant responses to reveal three core themes about alignment between design intent and reader interpretation, limitations of cued tasks for predicting real-world understanding, and the insufficiency of chart type alone to predict what people extract from a visualization. The findings show that designers’ stated objectives often do not fully reflect the patterns readers perceive, and that data type, composition, and supplemental cues substantially shape interpretation. The work argues for multi-perspective evaluation of visualization effectiveness, combining low-level cued tasks with high-level comprehension studies to better guide design guidelines and evaluation methodologies, with implications for practitioners and researchers seeking robust, accessible data communication. Overall, the study calls for broader, more nuanced design and assessment practices that reflect how diverse audiences actually interpret visualizations in the wild.
Abstract
Designers often create visualizations to achieve specific high-level analytical or communication goals. These goals require people to naturally extract complex, contextualized, and interconnected patterns in data. While limited prior work has studied general high-level interpretation, prevailing perceptual studies of visualization effectiveness primarily focus on isolated, predefined, low-level tasks, such as estimating statistical quantities. This study more holistically explores visualization interpretation to examine the alignment between designers' communicative goals and what their audience sees in a visualization, which we refer to as their comprehension. We found that statistics people effectively estimate from visualizations in classical graphical perception studies may differ from the patterns people intuitively comprehend in a visualization. We conducted a qualitative study on three types of visualizations -- line graphs, bar graphs, and scatterplots -- to investigate the high-level patterns people naturally draw from a visualization. Participants described a series of graphs using natural language and think-aloud protocols. We found that comprehension varies with a range of factors, including graph complexity and data distribution. Specifically, 1) a visualization's stated objective often does not align with people's comprehension, 2) results from traditional experiments may not predict the knowledge people build with a graph, and 3) chart type alone is insufficient to predict the information people extract from a graph. Our study confirms the importance of defining visualization effectiveness from multiple perspectives to assess and inform visualization practices.