DomusFM: A Foundation Model for Smart-Home Sensor Data

TL;DR

This paper presents DomusFM, the first foundation model pretrained directly on diverse smart-home sensor data to address data scarcity and privacy in ADL recognition and next-events prediction. The model uses a dual-contrastive, self-supervised framework to learn token-level semantic attributes and sequence-level temporal dependencies, via an event-level encoder and a contextualized transformer, with semantic embeddings derived from a lightweight language model. Through leave-one-dataset-out evaluation on seven public datasets, DomusFM achieves superior accuracy and generalization, even when fine-tuned with as little as 5% of labeled data, and demonstrates strong edge-deployability (memory < 500 MB, ~10 ms per window). Compared with zero-shot LLM approaches, DomusFM provides faster, private, and more scalable performance while maintaining high task versatility, making it practical for real-world smart-home systems and edge computing scenarios.

Abstract

Smart-home sensor data holds significant potential for several applications, including healthcare monitoring and assistive technologies. Existing approaches, however, face critical limitations. Supervised models require impractical amounts of labeled data. Foundation models for activity recognition focus only on inertial sensors, failing to address the unique characteristics of smart-home binary sensor events: their sparse, discrete nature combined with rich semantic associations. LLM-based approaches, while tested in this domain, still raise several issues regarding the need for natural language descriptions or prompting, and reliance on either external services or expensive hardware, making them infeasible in real-life scenarios due to privacy and cost concerns. We introduce DomusFM, the first foundation model specifically designed and pretrained for smart-home sensor data. DomusFM employs a self-supervised dual contrastive learning paradigm to capture both token-level semantic attributes and sequence-level temporal dependencies. By integrating semantic embeddings from a lightweight language model and specialized encoders for temporal patterns and binary states, DomusFM learns generalizable representations that transfer across environments and tasks related to activity and event analysis. Through leave-one-dataset-out evaluation across seven public smart-home datasets, we demonstrate that DomusFM outperforms state-of-the-art baselines on different downstream tasks, achieving superior performance even with only 5% of labeled training data available for fine-tuning. Our approach addresses data scarcity while maintaining practical deployability for real-world smart-home systems.

Figures (4)

• Figure 1: Event streams from the two different kinds of sensors.
• Figure 2: High level architecture
• Figure 3: Detailed architecture of DomusFM's two main components. The Event-level Feature Extraction encodes sensor event attributes through specialized encoders and integrates them via Attribute Self-Attention. The Contextualized Event-level Feature Extraction uses transformer layers to capture temporal dependencies and produce context-aware event representations.
• Figure 4: Dual contrastive learning framework for pretraining DomusFM. Attribute-level contrastive loss learns robust event representations by masking individual attributes, while event-level contrastive loss captures temporal context by masking entire events. Both stages use augmented data streams to learn representations through contrastive objectives.