Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Low-Power On-Device Gesture Recognition with Einsum Networks

Sahar Golipoor, Lingyun Yao, Martin Andraud, Stephan Sigg

TL;DR

The paper tackles ultra-low-power gesture recognition on distributed, resource-constrained devices by integrating ambient backscatter RFID sensing with Einsum Networks, a class of probabilistic circuits with tractable inference. It proposes on-device processing of RSS, phase, and AoA features across multiple constrained devices, followed by prediction fusion to recognize 21 gestures with high accuracy. The key contributions include three device-specific Einsum Network models trained on distinct feature sets, achieving 97.96% accuracy when fused, and a detailed computation-cost comparison showing substantial efficiency gains over deep neural networks. The work demonstrates practical potential for energy-efficient, edge-based RF sensing and gesture recognition with hardware-friendly probabilistic circuitry.

Abstract

We design a gesture-recognition pipeline for networks of distributed, resource constrained devices utilising Einsum Networks. Einsum Networks are probabilistic circuits that feature a tractable inference, explainability, and energy efficiency. The system is validated in a scenario of low-power, body-worn, passive Radio Frequency Identification-based gesture recognition. Each constrained device includes task-specific processing units responsible for Received Signal Strength (RSS) and phase processing or Angle of Arrival (AoA) estimation, along with feature extraction, as well as dedicated Einsum hardware that processes the extracted features. The output of all constrained devices is then fused in a decision aggregation module to predict gestures. Experimental results demonstrate that the method outperforms the benchmark models.

Low-Power On-Device Gesture Recognition with Einsum Networks

TL;DR

The paper tackles ultra-low-power gesture recognition on distributed, resource-constrained devices by integrating ambient backscatter RFID sensing with Einsum Networks, a class of probabilistic circuits with tractable inference. It proposes on-device processing of RSS, phase, and AoA features across multiple constrained devices, followed by prediction fusion to recognize 21 gestures with high accuracy. The key contributions include three device-specific Einsum Network models trained on distinct feature sets, achieving 97.96% accuracy when fused, and a detailed computation-cost comparison showing substantial efficiency gains over deep neural networks. The work demonstrates practical potential for energy-efficient, edge-based RF sensing and gesture recognition with hardware-friendly probabilistic circuitry.

Abstract

We design a gesture-recognition pipeline for networks of distributed, resource constrained devices utilising Einsum Networks. Einsum Networks are probabilistic circuits that feature a tractable inference, explainability, and energy efficiency. The system is validated in a scenario of low-power, body-worn, passive Radio Frequency Identification-based gesture recognition. Each constrained device includes task-specific processing units responsible for Received Signal Strength (RSS) and phase processing or Angle of Arrival (AoA) estimation, along with feature extraction, as well as dedicated Einsum hardware that processes the extracted features. The output of all constrained devices is then fused in a decision aggregation module to predict gestures. Experimental results demonstrate that the method outperforms the benchmark models.
Paper Structure (20 sections, 16 equations, 4 figures, 7 tables)

This paper contains 20 sections, 16 equations, 4 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. System and Signal Models
  4. System Model
  5. Signal Model
  6. Data Collection and Processing
  7. RSS and Phase processing
  8. AoA estimation
  9. Feature Extraction
  10. Accuracy Performance
  11. Probabilistic Approach using the Einsum network
  12. Comparison with other methods
  13. Computation Complexity Comparison
  14. Deep Neural network based methods
  15. Probabilistic Approach using Einsum network
  16. ...and 5 more sections

Figures (4)

  • Figure 1: Example structure of Probabilistic model: Probabilistic circuits; Leaf constrained devices $\Theta$ encodes distributions; $\otimes$ and $\oplus$ perform joint and mixture probability respectively.
  • Figure 2: The gestures performed by participants are (a) LD: Lateral Down (1), (b) LF: Lateral to Front (2), (c) LR: Lateral Raise (3), (d) LAC: Left Arm Circle (4), (e) RAC: Right Arm Circle (5), (f) L: Lift (6), (g) Pl: Pull (7), (h) Ps: Push (8), (i) LRo: Left Round (9), (j) RR: Right Round (10), (k) SL: Swipe Left (11), (l) SR: Swipe Right (12), (m) 2HLD: Two Hands Lateral Down (13), (n) 2HLF: Two Hands Lateral to Front (14), (o) 2HLR: Two Hands Lateral Raise (15), (p) 2HIC: Two Hands Inward Circle (16), (q) 2HOC: Two Hands Outward Circle (17), (r) 2HL: Two Hands Lift (18), (s) 2HPl: Two Hands Pull (19), (t) 2HPs: Two Hands Push (20), (u) 2HR: Two Hands Round (21).
  • Figure 3: Normalized confusion matrices for a single Einsum network using: (a) statistical features from RSS and phase; (b) statistical features extracted from AoA; (c) wavelet-based features derived from AoA.
  • Figure 4: Normalized confusion matrix obtained by averaging the outputs of constrained devices in the decision aggregation module (Merged Einsum Networks), achieving an accuracy of 97.96%