Neuro-Symbolic Fusion of Wi-Fi Sensing Data for Passive Radar with Inter-Modal Knowledge Transfer

TL;DR

This work presents DeepProbHAR, a neuro-symbolic framework that fuses Wi-Fi CSI signals from multiple antennas with declarative knowledge extracted from camera data to perform indoor human activity recognition. By compressing CSI via a variational autoencoder and combining six small neural nets with a DeepProbLog-based symbolic layer, the method achieves competitive HAR performance while providing interpretable, simple-movement classifiers derived without exhaustive labeling. The approach demonstrates that domain-dependent rules from an alternative modality can guide learning, improving transparency and data efficiency, and shows that multi-antenna fusion with the delayed fusion strategy yields the strongest results, approaching a video-based upper bound. The findings suggest a promising path for interpretable, data-efficient passive sensing in real-world indoor environments and motivate future work on unseen activities and robustness to distribution shifts.

Abstract

Wi-Fi devices, akin to passive radars, can discern human activities within indoor settings due to the human body's interaction with electromagnetic signals. Current Wi-Fi sensing applications predominantly employ data-driven learning techniques to associate the fluctuations in the physical properties of the communication channel with the human activity causing them. However, these techniques often lack the desired flexibility and transparency. This paper introduces DeepProbHAR, a neuro-symbolic architecture for Wi-Fi sensing, providing initial evidence that Wi-Fi signals can differentiate between simple movements, such as leg or arm movements, which are integral to human activities like running or walking. The neuro-symbolic approach affords gathering such evidence without needing additional specialised data collection or labelling. The training of DeepProbHAR is facilitated by declarative domain knowledge obtained from a camera feed and by fusing signals from various antennas of the Wi-Fi receivers. DeepProbHAR achieves results comparable to the state-of-the-art in human activity recognition. Moreover, as a by-product of the learning process, DeepProbHAR generates specialised classifiers for simple movements that match the accuracy of models trained on finely labelled datasets, which would be particularly costly.

Figures (9)

• Figure 1: Magnitude of the csi collected by the same antenna when a person performs two different activities, namely running (top) and clapping (bottom). csi values are dimensionless and are reported as measured by the Wi-Fi chipset.
• Figure 2: Sample of the video dataset for two different activities: a) walking and b) waving both hands. The key points in every video frame help to discern the right side (highlighted with coloured dots) from the left side of the candidate.
• Figure 3: The vae models map the input csi data onto the four parameters of a bivariate Gaussian distribution (mean $Z_\mu$ and variance $Z_{\sigma^2}$ along two axes), which we can be used as a compressed representation of the input csi.
• Figure 4: Computation of the right upper arm's angle $\alpha$. The same operation applies to all the other limb segments.
• Figure 5: The feature $\delta\xspace_l$ corresponding the right lower leg indicates the motion of that limb for each target activity.