Multi-modal Atmospheric Sensing to Augment Wearable IMU-Based Hand Washing Detection

TL;DR

This work tackles the reliability of hand-washing detection using wearables by augmenting IMU data with atmospheric sensing (humidity, temperature, pressure). The authors develop an open-source wrist-worn prototype based on Puck.js and a BME280 module, and collect a benchmark dataset from 10 participants (43 hand-wash events) to evaluate sensor contributions. Visual analyses show humidity rises during washing, suggesting potential contextual cues, while ML ablations indicate no clear performance gain from atmospheric sensors with basic features, though personalized models benefit more. The study provides open hardware, data, and code to spur further research, highlighting that humidity- and environment-aware features need more sophisticated modeling to translate into robust hand-wash detection in real-world settings.

Abstract

Hand washing is a crucial part of personal hygiene. Hand washing detection is a relevant topic for wearable sensing with applications in the medical and professional fields. Hand washing detection can be used to aid workers in complying with hygiene rules. Hand washing detection using body-worn IMU-based sensor systems has been shown to be a feasible approach, although, for some reported results, the specificity of the detection was low, leading to a high rate of false positives. In this work, we present a novel, open-source prototype device that additionally includes a humidity, temperature, and barometric sensor. We contribute a benchmark dataset of 10 participants and 43 hand-washing events and perform an evaluation of the sensors' benefits. Added to that, we outline the usefulness of the additional sensor in both the annotation pipeline and the machine learning models. By visual inspection, we show that especially the humidity sensor registers a strong increase in the relative humidity during a hand-washing activity. A machine learning analysis of our data shows that distinct features benefiting from such relative humidity patterns remain to be identified.

Figures (5)

• Figure 1: Overview of the wiring needed to attach the BME280 sensor board. The Puck.js provides power to the sensor which can be read via I²C using the D1 and D2 pins.
• Figure 2: Our prototype consists of a Puck.js with the attached BME280 sensor. A custom 3D-printed enclosure can be mounted on the wrist using the wristband.
• Figure 3: Statistics for the duration in seconds of all 43 recorded hand washes. The box plot shows the median (red solid line), mean (green dashed line), quartiles (box extents), and minimum and maximum (whiskers).
• Figure 4: Example accelerometer and humidity sensor plot of one hand washing (HW in legend) instance. We can observe the humidity rising once the participant starts washing their hands at time t = 0. Additionally, the signal RSSI received from a Bluetooth beacon placed at the sink is shown (navy blue). The Beacon signal was only used for labeling and is non-zero while the participant remained near the sink, e.g. to dry off the hands after washing.
• Figure 5: Response of the (a) humidity, (b) temperature, and (c) pressure sensors to hand washing, averaged (in dark blue) over all recorded hand washes, with bootstrapped 95% confidence interval (in light blue). The start of the hand washing is marked with a green vertical line. The yellow vertical lines mark the respective ends of all the handwashing instances.