Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Visual Validation versus Visual Estimation: A Study on the Average Value in Scatterplots

Daniel Braun, Ashley Suh, Remco Chang, Michael Gleicher, Tatiana von Landesberger

TL;DR

This study probes how people visually validate a simple statistical model (the mean) in scatterplots and how this compares to visual estimation. Through two between-subject online studies (volunteers and crowdsourced participants), the authors find that validation is less accurate than estimation but both are unbiased, and that the threshold for acceptance aligns with the boundary of the $95\%$ confidence interval. The work demonstrates that viewers’ perceived confidence intervals reflect a conventional statistical standard and offers a baseline for understanding visual validation, with implications for visualization design and further research across data representations. Overall, the findings reveal that people are consistent in validation yet conservative in deviation tolerance, informing how visualizations should present or precompute model fits. The results motivate extending the analysis beyond averages to broader models and settings in data visualization and decision support.

Abstract

We investigate the ability of individuals to visually validate statistical models in terms of their fit to the data. While visual model estimation has been studied extensively, visual model validation remains under-investigated. It is unknown how well people are able to visually validate models, and how their performance compares to visual and computational estimation. As a starting point, we conducted a study across two populations (crowdsourced and volunteers). Participants had to both visually estimate (i.e, draw) and visually validate (i.e., accept or reject) the frequently studied model of averages. Across both populations, the level of accuracy of the models that were considered valid was lower than the accuracy of the estimated models. We find that participants' validation and estimation were unbiased. Moreover, their natural critical point between accepting and rejecting a given mean value is close to the boundary of its 95% confidence interval, indicating that the visually perceived confidence interval corresponds to a common statistical standard. Our work contributes to the understanding of visual model validation and opens new research opportunities.

Visual Validation versus Visual Estimation: A Study on the Average Value in Scatterplots

TL;DR

This study probes how people visually validate a simple statistical model (the mean) in scatterplots and how this compares to visual estimation. Through two between-subject online studies (volunteers and crowdsourced participants), the authors find that validation is less accurate than estimation but both are unbiased, and that the threshold for acceptance aligns with the boundary of the confidence interval. The work demonstrates that viewers’ perceived confidence intervals reflect a conventional statistical standard and offers a baseline for understanding visual validation, with implications for visualization design and further research across data representations. Overall, the findings reveal that people are consistent in validation yet conservative in deviation tolerance, informing how visualizations should present or precompute model fits. The results motivate extending the analysis beyond averages to broader models and settings in data visualization and decision support.

Abstract

We investigate the ability of individuals to visually validate statistical models in terms of their fit to the data. While visual model estimation has been studied extensively, visual model validation remains under-investigated. It is unknown how well people are able to visually validate models, and how their performance compares to visual and computational estimation. As a starting point, we conducted a study across two populations (crowdsourced and volunteers). Participants had to both visually estimate (i.e, draw) and visually validate (i.e., accept or reject) the frequently studied model of averages. Across both populations, the level of accuracy of the models that were considered valid was lower than the accuracy of the estimated models. We find that participants' validation and estimation were unbiased. Moreover, their natural critical point between accepting and rejecting a given mean value is close to the boundary of its 95% confidence interval, indicating that the visually perceived confidence interval corresponds to a common statistical standard. Our work contributes to the understanding of visual model validation and opens new research opportunities.
Paper Structure (19 sections, 1 equation, 4 figures)

This paper contains 19 sections, 1 equation, 4 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Visual Model Estimation
  4. Visual Model Validation
  5. User Study
  6. Experimental Design
  7. Pilot Study
  8. Main Study
  9. Stimuli Design
  10. Experimental Setting & Participants
  11. Analysis
  12. Results
  13. Distribution and Bias
  14. Accuracy of Visual Validation vs. Visual Estimation
  15. Critical Point of Validation
  16. ...and 4 more sections

Figures (4)

  • Figure 1: Histogram of the deviations of the drawn lines in the estimation tasks for both populations.
  • Figure 2: Validation and estimation accuracy for both populations (absolute deviation). Blue line: Cumulative distribution for the estimation errors. Orange line: Logistic regression for the validation acceptance. Green line: Statistical 95% CI.
  • Figure 3: Comparison of the logistic regressions for the acceptance rates of positive and negative deviations of the volunteered population. Green line: statistical 95% CI.
  • Figure 4: Acceptance rate of the shown lines in the validation tasks (absolute deviation) of the volunteered population. Yellow line: The raw percentages per deviation. Orange line: The logistic regression of the yellow line's data. Green line: Statistical 95% CI.