Micro Hand Gesture Recognition System Using Ultrasonic Active Sensing

TL;DR

Micro hand gesture recognition enables touchless interaction, especially for wearables and automotive contexts. The paper presents HUG, an ultrasonic active-sensing system that yields high-resolution range-Doppler features, enabling time-sequence gesture classification. It offers two recognition pipelines: a lightweight state-transition-based HMM for efficiency and a higher-accuracy end-to-end neural network, achieving about 89% and 96.34% respectively on seven micro gestures. A real-time prototype demonstrates feasibility for practical use, highlighting potential for compact, energy-efficient wearable devices.

Abstract

In this paper, we propose a micro hand gesture recognition system and methods using ultrasonic active sensing. This system uses micro dynamic hand gestures for recognition to achieve human-computer interaction (HCI). The implemented system, called hand-ultrasonic gesture (HUG), consists of ultrasonic active sensing, pulsed radar signal processing, and time-sequence pattern recognition by machine learning. We adopt lower frequency (300 kHz) ultrasonic active sensing to obtain high resolution range-Doppler image features. Using high quality sequential range-Doppler features, we propose a state-transition-based hidden Markov model for gesture recognition. This method achieves a recognition accuracy of nearly 90\% by using symbolized range-Doppler features and significantly reduces the computational complexity and power consumption. Furthermore, to achieve higher classification accuracy, we utilize an end-to-end neural network model and obtain a recognition accuracy of 96.32\%. In addition to offline analysis, a real-time prototype is released to verify our method's potential for application in the real world.

Figures (10)

• Figure 1: Pipeline Design Block Diagram
• Figure 2: Platform Design
• Figure 3: Range-Doppler feature
• Figure 4: Range-Doppler feature images sampled in the "button off" (see Fig. \ref{['gestures']}) gesture. The first subfigure shows the index finger and thumb separated at a 3 cm distance. Noise will be removed using time smoothing to increase robustness to the 3 marked "noise" objects in the first subfigure. The second and third subfigures show the index finger moving down with an acceleration while the thumb is nearly motionless with only a tiny upwards velocity. The last subfigure shows two fingers being touched. Note that an object's trajectory will always be a curve in the range-Doppler image. The represented states are labeled at the center of the detected objects.
• Figure 5: State transition diagram. The states represent the types of scattering points, and the arrows denote the methods of transition. Here is an introduction to all the states: Initialization states consist of two states, namely, the state 'new' and the state '0', where state 'new' means the initial states, and state '0' represents uncertain states and the first missing scattering points states. Missing states include states '-1' and '-2', which reflect missing some frames' range-Doppler feature image's scattering points. Tracking states include states '1', '2', '3', and '4', which describe static and dynamic objects' tracking and locking. Ending states include '8' and 'del', which denote the end of motion when the dynamic object's speed is less than the threshold or more than 3 frames' scattering points are missing.