Dynamic User-controllable Privacy-preserving Few-shot Sensing Framework
Ajesh Koyatan Chathoth, Shuhao Yu, Stephen Lee
TL;DR
This work tackles privacy in IMU-based sensing by enabling dynamic, user-controlled protection of sensitive activities without model retraining. It introduces PrivCLIP, a three-component framework that combines IMU-CLIP for few-shot cross-modal activity detection, Privacy Personalizer for real-time policy control, and ACT-SANITIZER with IMU-GPT to sanitize data by replacing sensitive activity traces with semantically similar non-sensitive ones. The approach leverages multimodal contrastive learning to align IMU signals with natural language descriptions under a supervised loss $\mathcal{L}^{\text{sup}}$, enabling effective few-shot and zero-shot detection and safe data transformation. Empirical results on three HAR datasets show PrivCLIP outperforms baseline privacy methods in both privacy protection and data utility, while maintaining dynamic adaptability to user preferences. The framework thus offers a practical path to privacy-preserving, user-driven HAR in IoT and edge deployments.
Abstract
User-controllable privacy is important in modern sensing systems, as privacy preferences can vary significantly from person to person and may evolve over time. This is especially relevant in devices equipped with Inertial Measurement Unit (IMU) sensors, such as smartphones and wearables, which continuously collect rich time-series data that can inadvertently expose sensitive user behaviors. While prior work has proposed privacy-preserving methods for sensor data, most rely on static, predefined privacy labels or require large quantities of private training data, limiting their adaptability and user agency. In this work, we introduce PrivCLIP, a dynamic, user-controllable, few-shot privacy-preserving sensing framework. PrivCLIP allows users to specify and modify their privacy preferences by categorizing activities as sensitive (black-listed), non-sensitive (white-listed), or neutral (gray-listed). Leveraging a multimodal contrastive learning approach, PrivCLIP aligns IMU sensor data with natural language activity descriptions in a shared embedding space, enabling few-shot detection of sensitive activities. When a privacy-sensitive activity is identified, the system uses a language-guided activity sanitizer and a motion generation module (IMU-GPT) to transform the original data into a privacy-compliant version that semantically resembles a non-sensitive activity. We evaluate PrivCLIP on multiple human activity recognition datasets and demonstrate that it significantly outperforms baseline methods in terms of both privacy protection and data utility.