Unsupervised Domain Adaptation for RF-based Gesture Recognition

TL;DR

This article proposes an unsupervised domain adaptation framework for RF-based gesture recognition aiming to enhance the performance of the recognition model in new conditions by making effective use of the unlabeled data from new conditions and proposes pseudo labeling and consistency regularization.

Abstract

Human gesture recognition with Radio Frequency (RF) signals has attained acclaim due to the omnipresence, privacy protection, and broad coverage nature of RF signals. These gesture recognition systems rely on neural networks trained with a large number of labeled data. However, the recognition model trained with data under certain conditions would suffer from significant performance degradation when applied in practical deployment, which limits the application of gesture recognition systems. In this paper, we propose an unsupervised domain adaptation framework for RF-based gesture recognition aiming to enhance the performance of the recognition model in new conditions by making effective use of the unlabeled data from new conditions. We first propose pseudo-labeling and consistency regularization to utilize unlabeled data for model training and eliminate the feature discrepancies in different domains. Then we propose a confidence constraint loss to enhance the effectiveness of pseudo-labeling, and design two corresponding data augmentation methods based on the characteristic of the RF signals to strengthen the performance of the consistency regularization, which can make the framework more effective and robust. Furthermore, we propose a cross-match loss to integrate the pseudo-labeling and consistency regularization, which makes the whole framework simple yet effective. Extensive experiments demonstrate that the proposed framework could achieve 4.35% and 2.25% accuracy improvement comparing with the state-of-the-art methods on public WiFi dataset and millimeter wave (mmWave) radar dataset, respectively.

Figures (18)

• Figure 1: Overview of the framework. We first augment the raw unlabeled target domain data to obtain the augmented data, then the labeled data and unlabeled data are simultaneously fed into the model to obtain three groups of predictions. After obtaining the pseudo labels from the predictions of raw unlabeled target domain data, we adopt the pseudo labels as the ground truth of the predictions of the augmented data and compute the cross-match loss $L_{u}$ between them. Moreover, the supervised loss $L_{s}$ between the predictions of the source domain feature and labels is computed for model training, and a confidence constraint loss $L_{c}$ is adopted to enhance the effectiveness of pseudo-labeling. During training, the three losses are optimized simultaneously.
• Figure 2: Structure of gesture recognition model.
• Figure 3: The cross-match loss $L_{u}$ is obtained by integrating the self-supervised loss $L_{self}$ and the consistency regularization loss $L_{reg}$.
• Figure 4: Confidence Constraint Loss $L_{c}$: the Kullback-Leibler (KL) divergence between the predictions $\mathbf{\hat{y}}_{t}$ and the matrix $\mathbf{J}$.
• Figure 5: Our two data augmentation methods.