Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Gesture Recognition from body-Worn RFID under Missing Data

Sahar Golipoor, Richard T. Brophy, Ying Liu, Reza Ghazalian, Stephan Sigg

TL;DR

The paper addresses gesture recognition using body-worn passive RFID tags in the presence of missing data. It combines a data-processing pipeline (including interpolation, extrapolation, imputation, and proximity-based inference) with a temporal graph neural network that applies self-attention to exploit cross-tag correlations among eight EPC-tag nodes, represented as a tensor of shape $B \times T \times N \times D$ with $T=30$, $N=8$, $D=2$. The approach achieves up to $98.13\%$ accuracy for 21 gestures and $89.28\%$ under leave-one-person-out, outperforming state-of-the-art baselines across three environments and 17 subjects. The results also reveal the relative importance of tag placements (arms vs wrists) and demonstrate robustness to intermittent tag misdetections through the proposed imputation strategy.

Abstract

We explore hand-gesture recognition through the use of passive body-worn reflective tags. A data processing pipeline is proposed to address the issue of missing data. Specifically, missing information is recovered through linear and exponential interpolation and extrapolation. Furthermore, imputation and proximity-based inference are employed. We represent tags as nodes in a temporal graph, with edges formed based on correlations between received signal strength (RSS) and phase values across successive timestamps, and we train a graph-based convolutional neural network that exploits graph-based self-attention. The system outperforms state-of-the-art methods with an accuracy of 98.13% for the recognition of 21 gestures. We achieve 89.28% accuracy under leave-one-person-out cross-validation. We further investigate the contribution of various body locations on the recognition accuracy. Removing tags from the arms reduces accuracy by more than 10%, while removing the wrist tag only reduces accuracy by around 2%. Therefore, tag placements on the arms are more expressive for gesture recognition than on the wrist.

Gesture Recognition from body-Worn RFID under Missing Data

TL;DR

The paper addresses gesture recognition using body-worn passive RFID tags in the presence of missing data. It combines a data-processing pipeline (including interpolation, extrapolation, imputation, and proximity-based inference) with a temporal graph neural network that applies self-attention to exploit cross-tag correlations among eight EPC-tag nodes, represented as a tensor of shape with , , . The approach achieves up to accuracy for 21 gestures and under leave-one-person-out, outperforming state-of-the-art baselines across three environments and 17 subjects. The results also reveal the relative importance of tag placements (arms vs wrists) and demonstrate robustness to intermittent tag misdetections through the proposed imputation strategy.

Abstract

We explore hand-gesture recognition through the use of passive body-worn reflective tags. A data processing pipeline is proposed to address the issue of missing data. Specifically, missing information is recovered through linear and exponential interpolation and extrapolation. Furthermore, imputation and proximity-based inference are employed. We represent tags as nodes in a temporal graph, with edges formed based on correlations between received signal strength (RSS) and phase values across successive timestamps, and we train a graph-based convolutional neural network that exploits graph-based self-attention. The system outperforms state-of-the-art methods with an accuracy of 98.13% for the recognition of 21 gestures. We achieve 89.28% accuracy under leave-one-person-out cross-validation. We further investigate the contribution of various body locations on the recognition accuracy. Removing tags from the arms reduces accuracy by more than 10%, while removing the wrist tag only reduces accuracy by around 2%. Therefore, tag placements on the arms are more expressive for gesture recognition than on the wrist.
Paper Structure (11 sections, 10 equations, 8 figures, 1 table, 2 algorithms)

This paper contains 11 sections, 10 equations, 8 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Overview of the measurement System
  4. Data Preparation
  5. Interpolation
  6. Data Imputation
  7. Attention-Based Graph Learning
  8. Evaluation
  9. Parameter tuning
  10. Impact of tag placement and misdetection
  11. Conclusion

Figures (8)

  • Figure 1: The gestures performed by participants are (a) lateral raise (1), (b) push down (2), (c) lift (3), (d) pull (4), (e) push (5), (f) lateral to front (6), (g) swipe right (7), (h) swipe left (8), (i) throw (9), (j) arm swings (10), (k) two-hand throw (11), (l) two-hand push (12), (m) two-hand pull (13), (n) two-hand lateral raise (14), (o) left-arm circle (15), (p) right-arm circle (16), (q) two-hand outward circles (17), (r) two-hand inward circles (18), (s) two-hand lateral to front (19), (t) clockwise circle (20), and (u) counterclockwise circle (21).
  • Figure 2: Experimental setup. Eight passive RFID tags are attached to the subject, with phase and RSS data collected using an Impinj Speedway R420 reader and a circularly polarized antenna. Participants performed gestures at distances of 1.5m and 3m from the antenna.
  • Figure 3: Overview of the processing framework. Hardware-induced distortions are corrected, and missing data are addressed by stepwise interpolation and spatially guided data imputation.
  • Figure 4: Graph generation based on $(\rho_{i}, \varphi_{i})$ values across consecutive timestamps. For each EPC at a given timestamp, directed edges are formed by connecting to the nearest neighbors in the preceding timestamp using a temporal graph K-NN approach.
  • Figure 5: Probability of individual tag misdetection and joint tags' misdetection across all gesture executions.
  • ...and 3 more figures