Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hand Gesture Recognition from Doppler Radar Signals Using Echo State Networks

Towa Sano, Gouhei Tanaka

TL;DR

This paper addresses the need for efficient hand gesture recognition from Doppler FMCW radar signals by introducing an Echo State Network (ESN) framework with a parallel multi-reservoir architecture. Feature maps in time–space and time–frequency domains (RTM, DTM, and MDM where applicable) are processed by independent reservoirs, whose final states are concatenated for readout. The approach, evaluated on the Soli and Dop-NET datasets, achieves state-of-the-art or near-state-of-the-art accuracy with substantially lower training and inference costs than deep learning baselines, demonstrating strong potential for edge-enabled, real-time HGR. The results indicate that multi-reservoir ESNs can effectively fuse heterogeneous radar feature maps while mitigating interference, making them well-suited for resource-constrained HCI applications and robust deployment across subjects and sessions.

Abstract

Hand gesture recognition (HGR) is a fundamental technology in human computer interaction (HCI).In particular, HGR based on Doppler radar signals is suited for in-vehicle interfaces and robotic systems, necessitating lightweight and computationally efficient recognition techniques. However, conventional deep learning-based methods still suffer from high computational costs. To address this issue, we propose an Echo State Network (ESN) approach for radar-based HGR, using frequency-modulated-continuous-wave (FMCW) radar signals. Raw radar data is first converted into feature maps, such as range-time and Doppler-time maps, which are then fed into one or more recurrent neural network-based reservoirs. The obtained reservoir states are processed by readout classifiers, including ridge regression, support vector machines, and random forests. Comparative experiments demonstrate that our method outperforms existing approaches on an 11-class HGR task using the Soli dataset and surpasses existing deep learning models on a 4-class HGR task using the Dop-NET dataset. The results indicate that parallel processing using multi-reservoir ESNs are effective for recognizing temporal patterns from the multiple different feature maps in the time-space and time-frequency domains. Our ESN approaches achieve high recognition performance with low computational cost in HGR, showing great potential for more advanced HCI technologies, especially in resource-constrained environments.

Hand Gesture Recognition from Doppler Radar Signals Using Echo State Networks

TL;DR

This paper addresses the need for efficient hand gesture recognition from Doppler FMCW radar signals by introducing an Echo State Network (ESN) framework with a parallel multi-reservoir architecture. Feature maps in time–space and time–frequency domains (RTM, DTM, and MDM where applicable) are processed by independent reservoirs, whose final states are concatenated for readout. The approach, evaluated on the Soli and Dop-NET datasets, achieves state-of-the-art or near-state-of-the-art accuracy with substantially lower training and inference costs than deep learning baselines, demonstrating strong potential for edge-enabled, real-time HGR. The results indicate that multi-reservoir ESNs can effectively fuse heterogeneous radar feature maps while mitigating interference, making them well-suited for resource-constrained HCI applications and robust deployment across subjects and sessions.

Abstract

Hand gesture recognition (HGR) is a fundamental technology in human computer interaction (HCI).In particular, HGR based on Doppler radar signals is suited for in-vehicle interfaces and robotic systems, necessitating lightweight and computationally efficient recognition techniques. However, conventional deep learning-based methods still suffer from high computational costs. To address this issue, we propose an Echo State Network (ESN) approach for radar-based HGR, using frequency-modulated-continuous-wave (FMCW) radar signals. Raw radar data is first converted into feature maps, such as range-time and Doppler-time maps, which are then fed into one or more recurrent neural network-based reservoirs. The obtained reservoir states are processed by readout classifiers, including ridge regression, support vector machines, and random forests. Comparative experiments demonstrate that our method outperforms existing approaches on an 11-class HGR task using the Soli dataset and surpasses existing deep learning models on a 4-class HGR task using the Dop-NET dataset. The results indicate that parallel processing using multi-reservoir ESNs are effective for recognizing temporal patterns from the multiple different feature maps in the time-space and time-frequency domains. Our ESN approaches achieve high recognition performance with low computational cost in HGR, showing great potential for more advanced HCI technologies, especially in resource-constrained environments.
Paper Structure (20 sections, 6 equations, 8 figures, 6 tables)

This paper contains 20 sections, 6 equations, 8 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Radar-Based Hand Gesture Recognition
  4. Reservoir Computing Approaches
  5. Proposed Method
  6. Feature Extraction
  7. Echo State Network (ESN)
  8. Multi-Reservoir ESN
  9. Readout Classifier
  10. Experimental Setup
  11. Datasets
  12. ESN Configuration
  13. Evaluation Protocols
  14. Results and Discussion
  15. Classification Performance (50:50 Holdout Split And 10-Fold Cross Validation)
  16. ...and 5 more sections

Figures (8)

  • Figure 1: Examples of feature maps: (a) range-Doppler map (RDM), (b) range-time map (RTM) and (c) Doppler-time map (DTM).
  • Figure 2: Examples of micro-Doppler map (MDM).
  • Figure 3: Echo State Network architecture. The reservoir layer contains recurrently connected nodes with fixed random weights. Only the output layer weights $\mathbf{W}_{out}$ are trained.
  • Figure 4: Multi-reservoir ESN architecture. The $M$ independent reservoirs process RTMs and DTMs obtained from multiple antenna channels. Final states of all the reservoirs are concatenated and then fed into a readout classifier.
  • Figure 5: Set of hand gestures included in the Soli dataset wang2016interacting: (0) Pinch index, (1) Palm tilt, (2) Finger slider, (3) Pinch pinky, (4) Slow swipe, (5) Fast swipe, (6) Push, (7) Pull, (8) Finger rub, (9) Circle and (10) Palm hold. Sample images from the deep-soli GitHub repository (https://github.com/simonwsw/deep-soli), licensed under MIT License.
  • ...and 3 more figures