Average Estimates in Line Graphs Are Biased Toward Areas of Higher Variability

TL;DR

This paper demonstrates a robust bias in which average estimates from line graphs are overweighted toward regions of higher variability, a phenomenon termed variability overweighting. Through two preregistered experiments (n=140 and n=420), the authors show that the bias persists across encoding types but is reduced with dot-based representations and is modulated by arc-length salience. They develop a simple predictive model combining the data-average and the arc-average to account for the bias and demonstrate that arc-average adds explanatory power beyond the data-average. The findings have practical implications for visualization design, suggesting alternatives such as dot encodings or controlled encodings to mitigate biased inferences about averages and trends in time-series data.

Abstract

We investigate variability overweighting, a previously undocumented bias in line graphs, where estimates of average value are biased toward areas of higher variability in that line. We found this effect across two preregistered experiments with 140 and 420 participants. These experiments also show that the bias is reduced when using a dot encoding of the same series. We can model the bias with the average of the data series and the average of the points drawn along the line. This bias might arise because higher variability leads to stronger weighting in the average calculation, either due to the longer line segments (even though those segments contain the same number of data values) or line segments with higher variability being otherwise more visually salient. Understanding and predicting this bias is important for visualization design guidelines, recommendation systems, and tool builders, as the bias can adversely affect estimates of averages and trends.

Figures (11)

• Figure 1: First8of48stimuliforExperiment1.Thestimuliincludedtwovariabilitylevelsforeachseedandconditionswherethegraphsweremirroredcreatingstimuliwithvariabilityinthehigherandlowery-values.
• Figure 2: Examplestimulus.Participantscanmovethegrabberupanddowntowheretheyestimatetheline'saveragetobe.Eachseriesisshownasa500pxby200pxgraph.
• Figure 3: First8of144stimuliforExperiment2.Foreachseed,weincludedtwovariabilitylevels,threemarktypes,andstimuliwithvariabilityinthehigherandlowery-values.
• Figure 4: Countsoftheover-andunder-estimationsinExperiment1,brokendownbythevariabilityuppervs.lowercondition.
• Figure 5: Experiments1and2results,showingtheimpactofvariability,variabilityuppervs.lower,andmarktypeonestimationerror.TheleftpaneldetailsthefindingsofExperiment1withannotationsdescribingconfirmedhypotheses1-3.TherightpanelshowsExperiment2withannotationsdescribingconfirmedhypotheses4-5.Theblackbarswithinthedensityplotsshow95%CIswithameandot.