Causal Priors and Their Influence on Judgements of Causality in Visualized Data

TL;DR

A model is developed to capture the interaction between causal priors and visualized associations as they combine to impact a user's perceived causal relations and is suggested to help designers improve visualization design choices to better support visual causal inference.

Abstract

"Correlation does not imply causation" is a famous mantra in statistical and visual analysis. However, consumers of visualizations often draw causal conclusions when only correlations between variables are shown. In this paper, we investigate factors that contribute to causal relationships users perceive in visualizations. We collected a corpus of concept pairs from variables in widely used datasets and created visualizations that depict varying correlative associations using three typical statistical chart types. We conducted two MTurk studies on (1) preconceived notions on causal relations without charts, and (2) perceived causal relations with charts, for each concept pair. Our results indicate that people make assumptions about causal relationships between pairs of concepts even without seeing any visualized data. Moreover, our results suggest that these assumptions constitute causal priors that, in combination with visualized association, impact how data visualizations are interpreted. The results also suggest that causal priors may lead to over- or under-estimation in perceived causal relations in different circumstances, and that those priors can also impact users' confidence in their causal assessments. In addition, our results align with prior work, indicating that chart type may also affect causal inference. Using data from the studies, we develop a model to capture the interaction between causal priors and visualized associations as they combine to impact a user's perceived causal relations. In addition to reporting the study results and analyses, we provide an open dataset of causal priors for 56 specific concept pairs that can serve as a potential benchmark for future studies. We also suggest remaining challenges and heuristic-based guidelines to help designers improve visualization design choices to better support visual causal inference.

Figures (7)

• Figure 1: An instance of a visualization stimulus combined with a causal question and a confidence question from Study 2.
• Figure 2: This figure shows illustrations of the three chart types and five visualized association levels employed in Study 2. Left to right: scatterplots, line charts, and bar charts. Bottom to top: low association (around 0) to high association (around 1).
• Figure 3: Results of the simple slope analysis between causal priors and visualized associations. It shows that visualized associations would have a lower impact for higher causal priors. From top to bottom, the dark blue line shows the result for the mean-SD causal prior (2.01), blue shows the result for the mean causal prior (2.84), and light blue shows the result for the mean+SD causal prior (3.66).
• Figure 4: Results of the simple slope analysis between chart types and visualized associations. The values hint scatterplots have a lower impact on perceived causal relationships compared to bar charts and line charts. On the left (a) are the simple slope regressions, and on the right (b) are slopes at each interval.
• Figure 5: Results of user-reported confidence from Study 1. Concept pairs are ordered with increasing causal prior. The cyan lines indicate mean $\pm$ SD cyan lines, as with \ref{['fig:teaser']}.