Generalizing WiFi Gesture Recognition via Large-Model-Aware Semantic Distillation and Alignment

TL;DR

GLSDA tackles the limited generalization and semantic expressiveness of WiFi-based gesture recognition by leveraging large-model semantic priors through Large-Model Semantic Distillation (LSDM) and aligning cross-modal representations via Modality-Aligned Representation Optimization (MARO). LSDM transfers high-level gesture semantics from pretrained foundation models to the WiFi domain using weak vision-RF pairing and a contrastive distillation objective, while MARO enforces feature and classifier alignment to reduce cross-domain discrepancy and temporal instability. The approach, validated on Widar3.0, yields state-of-the-art generalization in in-domain and cross-domain settings with reduced model size and latency, evidencing the practical viability of semantic knowledge transfer for RF sensing. Overall, GLSDA provides a scalable, deployable framework for robust RF-based gesture interfaces in AIoT applications by bridging semantic grounding and cross-domain robustness.

Abstract

WiFi-based gesture recognition has emerged as a promising RF sensing paradigm for enabling non-contact and privacy-preserving human-computer interaction in AIoT environments. However, existing methods often suffer from limited generalization and semantic expressiveness due to the domain-sensitive nature of Channel State Information and the lack of high-level gesture abstraction. To address these challenges, we propose a novel generalization framework, termed Large-Model-Aware Semantic Distillation and Alignment (GLSDA), which leverages the semantic prior of pre-trained large foundation models to enhance gesture representation learning in both in-domain and cross-domain scenarios. Specifically, we first design a dual-path CSI encoding pipeline that captures geometric and dynamic gesture patterns via CSI-Ratio phase sequences and Doppler spectrograms. These representations are then fed into a Multiscale Semantic Encoder, which learns robust temporal embeddings and aligns them with gesture semantics through cross-modal attention mechanisms. To further enhance category discrimination, we introduce a Semantic-Aware Soft Supervision scheme that encodes inter-class correlations and reduces label ambiguity, especially for semantically similar gestures. Finally, we develop a Robust Dual-Distillation strategy to compress the aligned model into a lightweight student network, jointly distilling intermediate features and semantic-informed soft labels from the teacher model. Extensive experiments on the Widar3.0 benchmark show that GLSDA consistently outperforms state-of-the-art methods in both in-domain and cross-domain gesture recognition tasks, while significantly reducing model size and inference latency. Our method offers a scalable and deployable solution for generalized RF-based gesture interfaces in real-world AIoT applications.

Figures (3)

• Figure 1: Overview of the proposed GLSDA framework for WiFi-based gesture recognition. The system takes CSI sequences as input, which are preprocessed into complementary time-frequency and phase features. A pre-trained foundation model branch provides semantic embeddings from synchronized visual data, serving as a teacher for the Large-model Semantic Distillation Module (LSDM). The distilled semantic priors guide the WiFi encoder to learn semantically meaningful representations. Meanwhile, the Modality-Aligned Representation Optimization Module (MARO) further reduces cross-modal discrepancy and temporal inconsistency through feature alignment and classifier smoothing. Finally, the optimized WiFi representations are fed into a classifier for robust gesture recognition.
• Figure 2: Confusion Matrices Under Different Scenarios, where P&P, SW, CLP, SLD, D-O, and D-Z correspond to the gestures Push&Pull, Sweep, Clap, Slide, Draw-O, and Draw-ZigZag, respectively.
• Figure 3: Comparison: Baseline vs. Our Method