Could Humans Outshine AI in Visual Data Analysis?

TL;DR

The results suggest that, even when deviating from rationality, human gut reactions to visualizations can provide an advantage in inference and decision-making, with important implications for the design of visual analytics tools.

Abstract

People often use visualizations not only to explore a dataset but also to draw generalizable conclusions about underlying models or phenomena. While previous research has viewed deviations from rational analysis as problematic, we hypothesize that human reliance on non-normative heuristics may be advantageous in certain situations. In this study, we investigate scenarios where human intuition might outperform idealized statistical rationality. Our experiment assesses participants' accuracy in characterizing the parameters of known data-generating models from bivariate visualizations. Our findings show that, while participants generally demonstrated lower accuracy than statistical models, they often outperformed Bayesian agents, particularly when dealing with extreme samples. These results suggest that, even when deviating from rationality, human gut reactions to visualizations can provide an advantage. Our findings offer insights into how analyst intuition and statistical models can be integrated to improve inference and decision-making, with important implications for the design of visual analytics tools.

Figures (4)

• Figure 1: Elicitation Method. Participants utilize the first slider to adjust the correlation line (left). They then use the second slider to adjust the uncertainty range (right). Data points on the plot are refreshed at the frequency of 5 Hz to allow participants to see the model implication.
• Figure 2: Steps per trial that each participant. participants start by specifying initial belie of $\mu$ and $\sigma$ using the two sliders. They then observe the data sample and compare it against their prediction. Subsequently, they specify their posterior beliefs using the same interface as the first step.
• Figure 3: Comparison between humans and statistical machines in inferring true $\mu$, contingent on sample extremeness. The top histogram illustrates the empirical distribution observed at various $\Delta R$ levels.
• Figure 4: The sensitivity of the three agents to sample extremeness. The regression line shows model estimates. Points depict the observed, empirical responses. A weaker correlation implies more resilience to spurious samples.