Privacy-Preserving Covert Communication Using Encrypted Wearable Gesture Recognition

TL;DR

This work tackles privacy risks in wearable gesture-based covert communication by proposing an end-to-end system that performs gesture recognition entirely on encrypted data using a multi-party homomorphic learning pipeline. The approach eliminates exposure of raw sensor signals, intermediate features, and model outputs to untrusted infrastructure, enabling secure inference on both cloud and edge devices via CrypTen-based secure computation. A compact three-layer homomorphic neural network is trained and evaluated across heterogeneous hardware, demonstrating classification accuracy above 92% and practical latency on modern edge platforms. The system integrates two covert feedback modalities (haptic and visual) and validates feasibility with real-world data collection on commodity smartwatches, highlighting edge deployability, cryptographic privacy guarantees, and applicability to safety-critical and assistive scenarios. Overall, the work bridges encrypted machine learning and wearable security to deliver private, covert, and usable gesture-based communication in adversarial environments.

Abstract

Secure communication is essential in covert and safety-critical settings where verbal interactions may expose user intent or operational context. Wearable gesture-based communication enables low-effort, nonverbal interaction, but existing systems leak motion data, intermediate representations, or inference outputs to untrusted infrastructure, enabling intent inference, behavioral biometric leakage, and insider attacks. This work proposes a privacy-preserving gesture-based covert communication system that ensures, no raw sensor signals, learned features, or classification outputs are exposed to any third-party. The system employs a multi-party homomorphic learning pipeline for gesture recognition directly over encrypted motion data, preventing adversaries from inferring gesture semantics, replaying sensor traces, or accessing intermediate representations. To our knowledge, this work is the first to apply encrypted gesture recognition in a wearable-based covert communication setting. We design and evaluate haptic and visual feedback mechanisms for covert signal delivery and evaluate the system using 600 gesture samples from a commodity smartwatch, achieving over 94.44% classification accuracy and demonstrating the feasibility of the proposed system with practical deployability from high-performance systems to resource-constrained edge devices.

Figures (10)

• Figure 1: System overview.
• Figure 2: Example of temporal segmentation on a continuous wrist-motion stream. Highlighted windows indicate detected gesture-symbol segments (e.g., B, C, A), separated by pause.
• Figure 3: State-machine model for temporally segmented gesture communication. Opening and closing pauses delimit the active inference window.
• Figure 4: Example wrist-motion gyroscope traces illustrating the temporal segmentation protocol. Extended pauses indicate gesture boundaries, while motion segments correspond to individual gesture symbols.
• Figure 5: ROC curve of general neural network trained on different devices