Breaking Coordinate Overfitting: Geometry-Aware WiFi Sensing for Cross-Layout 3D Pose Estimation

TL;DR

This work identifies coordinate overfitting as a fundamental challenge in WiFi-based 3D pose estimation, where models memorize transceiver layouts rather than human motion. It introduces PerceptAlign, a geometry-conditioned framework that unifies WiFi and vision into a shared 3D space via a lightweight coordinate unification and encodes transceiver geometry as conditional priors fused with CSI using a Transformer-based backbone. The approach yields robust, layout-invariant pose estimation and achieves large gains over state-of-the-art baselines across diverse cross-domain settings on a newly built large-scale dataset. The results suggest geometry-aware learning as a viable path toward scalable, privacy-preserving WiFi sensing in real-world deployments.

Abstract

WiFi-based 3D human pose estimation offers a low-cost and privacy-preserving alternative to vision-based systems for smart interaction. However, existing approaches rely on visual 3D poses as supervision and directly regress CSI to a camera-based coordinate system. We find that this practice leads to coordinate overfitting: models memorize deployment-specific WiFi transceiver layouts rather than only learning activity-relevant representations, resulting in severe generalization failures. To address this challenge, we present PerceptAlign, the first geometry-conditioned framework for WiFi-based cross-layout pose estimation. PerceptAlign introduces a lightweight coordinate unification procedure that aligns WiFi and vision measurements in a shared 3D space using only two checkerboards and a few photos. Within this unified space, it encodes calibrated transceiver positions into high-dimensional embeddings and fuses them with CSI features, making the model explicitly aware of device geometry as a conditional variable. This design forces the network to disentangle human motion from deployment layouts, enabling robust and, for the first time, layout-invariant WiFi pose estimation. To support systematic evaluation, we construct the largest cross-domain 3D WiFi pose estimation dataset to date, comprising 21 subjects, 5 scenes, 18 actions, and 7 device layouts. Experiments show that PerceptAlign reduces in-domain error by 12.3% and cross-domain error by more than 60% compared to state-of-the-art baselines. These results establish geometry-conditioned learning as a viable path toward scalable and practical WiFi sensing.

Figures (9)

• Figure 1: Left: Conventional pipelines implicitly memorize the geometric layout of WiFi devices, conflating it with target knowledge and leading to coordinate overfitting. Right: PerceptAlign explicitly makes the model aware that WiFi transceiver geometry is a conditional factor rather than knowledge to be memorized.
• Figure 2: Sensing in vision and WiFi. The Vision system establishes an absolute world coordinate system rooted in the checkerboard $B$. The WiFi system establishes relative sensing coordinate system based on the transceiver locations. The colored ellipsoids represent the Fresnel Zones corresponding to each TX-RX pairs.
• Figure 3: System overview.
• Figure 4: Geometry-conditioned learning network architecture.
• Figure 5: Scenes and device placements used for data collection. Scene_3 contains multiple layouts (A/B/C) as documented in the dataset README file.