Mind Drifts, Data Shifts: Utilizing Mind Wandering to Track the Evolution of User Experience with Data Visualizations

TL;DR

This work addresses the gap in understanding dynamic cognition during data visualization by employing mind wandering as an in-situ measure of user experience. It combines a 100-stimulus dataset, a 45-second observation window, and post-view assessments to quantify how mind wandering interacts with trust, engagement, design quality, and recall. The study finds that mind wandering generally impairs short-term recall and trust, with certain design elements amplifying or mitigating MW and with MW serving as a partial mediator between encoding features and post-view outcomes. These findings suggest design guidelines to minimize cognitive load and current measurement approaches to incorporate dynamic cognitive states, advancing practical, real-time evaluation of visualization interfaces.

Abstract

User experience in data visualization is typically assessed through post-viewing self-reports, but these overlook the dynamic cognitive processes during interaction. This study explores the use of mind wandering -- a phenomenon where attention spontaneously shifts from a primary task to internal, task-related thoughts or unrelated distractions -- as a dynamic measure during visualization exploration. Participants reported mind wandering while viewing visualizations from a pre-labeled visualization database and then provided quantitative ratings of trust, engagement, and design quality, along with qualitative descriptions and short-term/long-term recall assessments. Results show that mind wandering negatively affects short-term visualization recall and various post-viewing measures, particularly for visualizations with little text annotation. Further, the type of mind wandering impacts engagement and emotional response. Mind wandering also functions as an intermediate process linking visualization design elements to post-viewing measures, influencing how viewers engage with and interpret visual information over time. Overall, this research underscores the importance of incorporating mind wandering as a dynamic measure in visualization design and evaluation, offering novel avenues for enhancing user engagement and comprehension.

Figures (10)

• Figure 1: Examples of visualization stimuli 10294209 that trigger early and more frequent reports of different types of mind-wandering.
• Figure 2: Study procedure: (a) Phase 1: 50 trials, tests at-a-glance comprehension. (b--d) Phase 2: 50 trials, mind-wandering self-reports d2016attending, collect post-viewing measures, short-term recall. (e) Phase 3: 1 trial, long-term recall tested with a visual recognition task.
• Figure 3: Scatterplot denoting the Regression Coefficient ($\beta$) results (y-axis) for observed/latent variables for mind wandering (MW) vs. collected quantitative post-viewing measures (x-axis). Color/shape denotes the mind wandering variable considered. We observe that all the 'relevant mind-wandering' variables cluster close together, as do irrelevant mind-wandering frequency, earliest occurrence, and aggregated frequency.
• Figure 4: Correlogram denoting how mind wandering (MW) affects the normalized valence (color) and strength (size) of adjective descriptions. We note that irrelevant MW causes the greatest decrease in adjective strength. MW on chart appearance increases positive valence of aesthetic appeal, irrelevant MW and earliest occurrence of MW increases negative valence of communicative utility. MW on chart data/topic has the weakest effects overall.
• Figure 5: Example of how short-term recall is coded for visualization takeaways and most salient visualization elements. (Gen: Generality Score).