(Un)fair devices: Moving beyond AI accuracy in personal sensing

TL;DR

This work advocates for a shift from prioritizing performance-oriented evaluations of personal devices to adopting assessments grounded in a human-centered approach, and provides guidelines for the design, development, evaluation, and use of unbiased AI in personal devices.

Abstract

Personal devices are omnipresent in our lives, seamlessly monitoring our activities, from smart rings tracking sleep patterns to smartwatches keeping an eye on missed heartbeats. The rich data streams from such devices fuel advanced Artificial Intelligence (AI) applications. Instead of solely relying on direct sensor measurements, these applications are increasingly leveraging Machine Learning (ML) model estimates to derive insights. But are these estimates biased or not? This literature review delivers compelling evidence about the impact of hidden biases that creep into ML models deployed on personal devices. We discuss critical bias issues drawn from prior work such as racial bias in pulse oximeters, weight bias in optical heart rate sensors, and sex bias in audio-based diagnostics. In response to these challenges, we advocate for a shift from prioritizing performance-oriented evaluations of personal devices to adopting assessments grounded in a human-centered approach. To facilitate this transition, we provide guidelines for the design, development, evaluation, and use of unbiased AI in personal devices, recognizing their potential impact on improving our health, lifestyle, and productivity -- more than any other technology.

Figures (6)

• Figure 1: Motivating examples of biased personal devices drawn from prior work. Research efforts have surfaced racial biases in (a) pulse oximetry fawzy2022racial and (b) optical heart rate sensors bent2020investigating, (c) sex bias in acoustic signals 10.1145/3534595, and (d) age bias in accelerometer and gyroscope measurements 10.1145/3351281.
• Figure 2: Conceptual differences between ML fairness and personal devices data. Biases are harder to surface, given the complexity of temporal data. Continuous data collection can introduce data drifts, resulting in larger and more dynamic datasets. Additionally, the ML tasks addressed by the personal devices community diverge from those typically explored in fairness research.
• Figure 3: Illustration of the literature synthesis methodology. A high-level overview of the process, including the querying, manual extraction, and filtering, and one of the main results. Note that for readability purposes, we present a simplified version of our query and data extraction. Our query retrieves $\sim58\%$ of all IMWUT publications, while our eligibility assessment filtering ends up with 89 papers ($\sim9\%$ of retrieved papers). We notice that only a very small fraction of all IMWUT papers looks at fairness issues, with only a small deviation across years.
• Figure 4: The query utilized for recovering relevant papers from the ACM Digital Library. Terms related to mobile, wearable, and ubiquitous computing are highlighted in green, ML in orange, and fairness in purple.
• Figure 5: PRISMA flow diagram for paper inclusion. Out of the 957 papers retrieved by our query after the screening, only 9% ($N=89$) did not check any exclusion criterion and thus were included in the literature synthesis.