Beyond Physical Labels: Redefining Domains for Robust WiFi-based Gesture Recognition

TL;DR

This work addresses the limited effectiveness of adversarial generalization in WiFi-based gesture recognition by redefining domain structure through data-driven latent domains. It introduces GesFi, which first denoises and standardizes CSI-derived inputs, then iteratively discovers latent domains from the data distribution and aligns them adversarially to learn domain-invariant gesture representations. Across three public datasets and real-world deployments, GesFi achieves substantial cross-domain improvements over state-of-the-art baselines, validating the value of latent-domain mining over traditional physical-domain partitioning. The results highlight the potential of combining data-driven domain discovery with adversarial learning to build robust, transferable WiFi sensing systems, while also pointing to opportunities for incorporating physics-informed modeling to further mitigate manifold distortions under large domain shifts.

Abstract

In this paper, we propose GesFi, a novel WiFi-based gesture recognition system that introduces WiFi latent domain mining to redefine domains directly from the data itself. GesFi first processes raw sensing data collected from WiFi receivers using CSI-ratio denoising, Short-Time Fast Fourier Transform, and visualization techniques to generate standardized input representations. It then employs class-wise adversarial learning to suppress gesture semantic and leverages unsupervised clustering to automatically uncover latent domain factors responsible for distributional shifts. These latent domains are then aligned through adversarial learning to support robust cross-domain generalization. Finally, the system is applied to the target environment for robust gesture inference. We deployed GesFi under both single-pair and multi-pair settings using commodity WiFi transceivers, and evaluated it across multiple public datasets and real-world environments. Compared to state-of-the-art baselines, GesFi achieves up to 78% and 50% performance improvements over existing adversarial methods, and consistently outperforms prior generalization approaches across most cross-domain tasks.

Figures (11)

• Figure 1: Prior approaches jiang2018towards typically assume that distributional shifts can be captured by physical labels, such as environments. However, we argue that physical domains are insufficient to represent cross-domain distributional variations in WiFi sensing. For example, within a single environment, a push gesture performed in opposite directions can result in entirely different CSI patterns, forming multiple disconnected distributional clusters. Conversely, push and pull gestures executed in different environments may exhibit highly similar CSI representations, leading to distributional confusion. In some cases, the variation in CSI within a single physical domain may exceed that between different domains.
• Figure 2: The domain adversarial learning based generalization framework.
• Figure 3: Exploratory experiments results. S-label and CF-label represent cross domain with physical domain labels and counter-intuitive physical domain labels, respectively.
• Figure 4: The system framework of GesFi.
• Figure 5: The WiFi latent domain mining based gesture recognition network.