A Multidimensional Assessment Method for Situated Visualization Understanding (MdamV)

TL;DR

This paper introduces MdamV, a Multidimensional Assessment Method for Situated Visualization Understanding, to capture how general audiences read, interpret, and critique visualizations beyond task performance. Grounded in learning sciences, MdamV spans six dimensions—Comprehending, Decoding, Aestheticizing, Critiquing, Reading, Contextualizing—and combines task performance with self-perceived abilities and open-ended critique. The method was validated through a representative Austrian survey (N=$438$) using static climate visualizations (line and bar charts), revealing notable deficits in comprehension of units, unfamiliarity with chart types, and significant correlations between self-assessed numeracy and data reading performance ($p=0.0004$ and $p=0.004$ in related analyses). Results demonstrate the value of assessing visualization understanding as a situated process tied to specific visuals, with implications for design, communication, and future methodological expansion. The study also discusses limitations and outlines pathways for broader applicability, including additional visualization formats and domain adaptations, as well as potential educational feedback components.

Abstract

How audiences read, interpret, and critique data visualizations is mainly assessed through performance tests featuring tasks like value retrieval. Yet, other factors shown to shape visualization understanding, such as numeracy, graph familiarity, and aesthetic perception, remain underrepresented in existing instruments. To address this, we design and test a Multidimensional Assessment Method of Situated Visualization Understanding (MdamV). This method integrates task-based measures with self-perceived ability ratings and open-ended critique, applied directly to the visualizations being read. Grounded in learning sciences frameworks that view understanding as a multifaceted process, MdamV spans six dimensions: Comprehending, Decoding, Aestheticizing, Critiquing, Reading, and Contextualizing. Validation was supported by a survey (N=438) representative of Austria's population (ages 18-74, male/female split), using a line chart and a bar chart on climate data. Findings show, for example, that about a quarter of respondents indicate deficits in comprehending simple data units, roughly one in five people felt unfamiliar with each chart type, and self-assessed numeracy was significantly related to data reading performance (p=0.0004). Overall, the evaluation of MdamV demonstrates the value of assessing visualization understanding beyond performance, framing it as a situated process tied to particular visualizations.

Figures (5)

• Figure 1: Overview of MdamV's structure with six assessment dimensions (Dim1–Dim6), alongside demographic questions. For the full question catalog, see the supplementary material.
• Figure 2: Data visualizations used in the MdamV survey with a German-speaking audience. Figure (a) shows an official English version of the tested line chart, while Figure (b) displays the original bar chart used in the survey, which is only available in German. See Sec. \ref{['climate-data-vis']} for details.
• Figure 3: Data reading tasks (Q5a-Q5d) were presented in true-or-false mode (see Fig. \ref{['fig:vis1+vis2']}). (a) shows correct vs. false responses and (b) the distribution of correct answers per chart, both with N=438.
• Figure 4: Correlation tables for (a) MdamV Dimension 1 to 5 and (b) questions within Dimensions 1 to 4 per surveyed chart. Indicated relations are based on Pearson's r, and all correlations are statistically significant ($p < 0.05$). See supplementary Fig. S1–S3 for more detailed views.
• Figure 5: Ordinal regression supports the observed relationship between age, self-assessed numeracy, and data reading task performance.