The Data-Expectation Gap: A Vocabulary Describing Experiential Qualities of Data Inaccuracies in Smartwatches

TL;DR

The paper tackles the data-expectation gap in smartwatch data, where users' experiences diverge from device outputs. It develops a context-rich vocabulary, grounded in two qualitative studies (online reviews and in-the-wild field work), to describe how mismatches emerge and generate tensions. The resulting taxonomy partitions the gap into mismatch types, contextualisation factors (temporality, activity, location, co-experience, emotions), and personal evaluation (trust, goals, history, explainability), with practical guidelines for designing tension-aware HDI. This work advances Personal Informatics and HDI by moving beyond traditional accuracy metrics to account for lived experiences, enabling more nuanced design and analysis of data-inaccuracies in wearables and beyond.

Abstract

Many users of wrist-worn wearable fitness trackers encounter the data-expectation gap - mismatches between data and expectations. While we know such discrepancies exist, we are no closer to designing technologies that can address their negative effects. This is largely because encounters with mismatches are typically treated unidimensionally, while they may differ in context and implications. This treatment does not allow the design of human-data interaction (HDI) mechanisms accounting for temporal, social, emotional, and other factors potentially influencing the perception of mismatches. To address this problem, we present a vocabulary that describes the breadth and context-bound character of encounters with the data-expectation gap, drawing from findings from two studies. Our work contributes to Personal Informatics research providing knowledge on how encounters with the data-expectation gap are embedded in people's daily lives, and a vocabulary encapsulating this knowledge, which can be used when designing HDI experiences in wearable fitness trackers.

Figures (12)

• Figure 1: The process we followed to construct the vocabulary.
• Figure 2: The smartwatches that we considered in the detailed analysis of online reviews. Fitbit Sense, which is not shown in the image, is very similar in appearance to Fitbit Sense 2. The figures were retrieved by Fitbit Fitbit2023 and Garmin Garmin2023.
• Figure 3: Fitbit Inspire 3, which was used in the field study (Figure retrieved by Fitbit Fitbit2023).
• Figure 4: The discussed scenarios of mismatches. F: data-feeling mismatch. L: data-logic mismatch.
• Figure 5: Evidence of the data-expectation gap in SQ.