Do You "Trust" This Visualization? An Inventory to Measure Trust in Visualizations

TL;DR

Trust in visualizations is variably defined and measured across fields, hindering cross-study synthesis. The authors adopt a data-driven approach to define trust in visualizations, compiling 113 items and using exploratory factor analysis to reveal three factors: Trust in Information, Clarity/Usability/Likability, and Individual Characteristics. They derive a compact eight-item inventory (four visualization-specific items plus four general-trust items) with strong reliability (ω ≈ 0.79 and 0.70) and demonstrate criterion validity via trust games with real stakes. The work provides a practical, adaptable measurement tool and guidance for applying trust measures across visualization research contexts, enabling more consistent comparisons and investigations of design and task effects on trust.

Abstract

Trust plays a critical role in visual data communication and decision-making, yet existing visualization research employs varied trust measures, making it challenging to compare and synthesize findings across studies. In this work, we first took a bottom-up, data-driven approach to understand what visualization readers mean when they say they "trust" a visualization. We compiled and adapted a broad set of trust-related statements from existing inventories and collected responses to visualizations with varying degrees of trustworthiness. Through exploratory factor analysis, we derived an operational definition of trust in visualizations. Our findings indicate that people perceive a trustworthy visualization as one that presents credible information and is comprehensible and usable. Building on this insight, we developed an eight-item inventory: four core items measuring trust in visualizations and four optional items controlling for individual differences in baseline trust tendency. We established the inventory's internal consistency reliability using McDonald's omega, confirmed its content validity by demonstrating alignment with theoretically-grounded trust dimensions, and validated its criterion validity through two trust games with real-world stakes. Finally, we illustrate how this standardized inventory can be applied across diverse visualization research contexts. Utilizing our inventory, future research can examine how design choices, tasks, and domains influence trust, and how to foster appropriate trusting behavior in human-data interactions.

Figures (5)

• Figure 1: The four visualization-related questions in our inventory measure trust in visualizations by capturing visualization clarity, usability, data credibility, and information comprehensibility. The map on the right is rated as more trustworthy because participants perceive its data as more reliable, find it easier to use, and consider it more comprehensible.
• Figure 2: Mean trust ratings for the creator, data, and design dimensions across the 13 visualizations selected from the pilot for the main experiment. Red and blue points indicate the minimum and maximum mean trust ratings observed for any visualization within each dimension. The selected visualizations encompass a diverse range of trust ratings, with some consistently high or low across all dimensions, while others exhibit mixed trust profiles---high in some dimensions and low in others.
• Figure 3: Comparison of factor models based on empirical BIC and root mean residual. Both metrics exhibit an "elbow" at the three-factor model, suggesting that it provides a favorable balance between model fit and parsimony.
• Figure 4: Distributions of resource allocations in the trust game. In both investment contexts, Dashboard A receives significantly more resources, indicating higher behavioral trust among participants.
• Figure 5: Viewer trust in visualizations as measured by the four visualization-related items in our inventory. Scores on negative items (#14 and #43) are reversed for clarity, ensuring that higher ratings consistently indicate greater trust. Dashed lines represent mean ratings. In both investment contexts, Dashboard A receives more resource allocations, supporting the criterion validity of our inventory in capturing behavioral trust.