Eating Speed Measurement Using Wrist-Worn IMU Sensors Towards Free-Living Environments

TL;DR

A temporal convolutional network combined with a multi-head attention module (TCN-MHA) is developed to detect bites (including eating and drinking gestures) from IMU data, which are clustered into eating episodes to advance eating speed measurement in near-free-living environments automatically and objectively.

Abstract

Eating speed is an important indicator that has been widely investigated in nutritional studies. The relationship between eating speed and several intake-related problems such as obesity, diabetes, and oral health has received increased attention from researchers. However, existing studies mainly use self-reported questionnaires to obtain participants' eating speed, where they choose options from slow, medium, and fast. Such a non-quantitative method is highly subjective and coarse at the individual level. This study integrates two classical tasks in automated food intake monitoring domain: bite detection and eating episode detection, to advance eating speed measurement in near-free-living environments automatically and objectively. Specifically, a temporal convolutional network combined with a multi-head attention module (TCN-MHA) is developed to detect bites (including eating and drinking gestures) from IMU data. The predicted bite sequences are then clustered into eating episodes. Eating speed is calculated by using the time taken to finish the eating episode to divide the number of bites. To validate the proposed approach on eating speed measurement, a 7-fold cross validation is applied to the self-collected fine-annotated full-day-I (FD-I) dataset, and a holdout experiment is conducted on the full-day-II (FD-II) dataset. The two datasets are collected from 61 participants with a total duration of 513 h, which are publicly available. Experimental results show that the proposed approach achieves a mean absolute percentage error (MAPE) of 0.110 and 0.146 in the FD-I and FD-II datasets, respectively, showcasing the feasibility of automated eating speed measurement in near-free-living environments.

Figures (13)

• Figure 1: Examples of wrist-worn IMU data collection. Two IMU sensors were mounted on both hands. Participants completed their daily activities without restriction. This figure only shows the food intake related scenes, their other daily activities, such as studying, walking, talking, running, cooking, were recorded by IMU sensors as well.
• Figure 2: (a) the data collection protocol; (b) the annotation method, the video label is for state 1, and the all '0' label is for state 2.
• Figure 3: The pie chart for eating styles (a) and eating locations (b). The eating speeds represent the average speed of each category.
• Figure 4: The violin plot of ground truth eating speed distribution among different datasets.
• Figure 5: The quantity ratio of eating gestures using only dominant hand on FD-I dataset. A ratio with 0.5 means half number of eating gestures are from the dominant hand, the others are from the non-dominant hand.