Self-supervised New Activity Detection in Sensor-based Smart Environments

TL;DR

CLAN addresses open-world novelty detection in sensor-based HAR by learning invariant, discriminative representations through a two-tower architecture that jointly analyzes time and frequency domains. It introduces Customized Strong Transformations (CST) to generate diverse, dataset-tailored negatives and applies multi-type contrastive learning with an auxiliary augmentation-type classifier, achieving strong AUROC gains (average around 9–15 percentage points above baselines) and robust performance across one-class and multi-class settings. The method demonstrates improved separation between known and new activities and maintains efficiency in inference, suggesting practical applicability in real-world smart environments. The work also emphasizes the importance of dataset-specific augmentation strategies and multi-perspective representations for reliable novelty detection in heterogeneous HAR contexts.

Abstract

With the rapid advancement of ubiquitous computing technology, human activity analysis based on time series data from a diverse range of sensors enables the delivery of more intelligent services. Despite the importance of exploring new activities in real-world scenarios, existing human activity recognition studies generally rely on predefined known activities and often overlook detecting new patterns (novelties) that have not been previously observed during training. Novelty detection in human activities becomes even more challenging due to (1) diversity of patterns within the same known activity, (2) shared patterns between known and new activities, and (3) differences in sensor properties of each activity dataset. We introduce CLAN, a two-tower model that leverages Contrastive Learning with diverse data Augmentation for New activity detection in sensor-based environments. CLAN simultaneously and explicitly utilizes multiple types of strongly shifted data as negative samples in contrastive learning, effectively learning invariant representations that adapt to various pattern variations within the same activity. To enhance the ability to distinguish between known and new activities that share common features, CLAN incorporates both time and frequency domains, enabling the learning of multi-faceted discriminative representations. Additionally, we design an automatic selection mechanism of data augmentation methods tailored to each dataset's properties, generating appropriate positive and negative pairs for contrastive learning. Comprehensive experiments on real-world datasets show that CLAN achieves a 9.24% improvement in AUROC compared to the best-performing baseline model.

Figures (12)

• Figure 1: Challenges faced by the existing novelty detection methods when applied to human activities.
• Figure 2: Illustration of CLAN for new activity detection in sensor-based smart environments.
• Figure 3: Visualization of examples for each dataset. The x-axis represents timestamps and the y-axis represents z-score normalized sensor values.
• Figure 4: ROC curve graphs for CLAN and the top-2 best-performing baselines in CASAS. ROC curves closer to the top-left corner indicate superior performance. The numerical labels on each ROC curve correspond to the known activity number. The Multi label represents the performance in multi-class scenarios.
• Figure 5: AUROC (%) in ARAS when one activity is designated as a known activity and another as a new activity. Brighter colors indicate better performance.