Generative Resident Separation and Multi-label Classification for Multi-person Activity Recognition

TL;DR

The paper tackles multi-person activity recognition with ambient sensors by contrasting resident separation (Seq2Res) and multi-label classification (BiGRU+Q2L), plus a two-stage combination. Seq2Res uses a Bahdanau-attention Seq2Seq framework to generate resident-specific event sequences, while BiGRU+Q2L employs a BiGRU encoder with a Query2Label Transformer decoder for joint label prediction. On the CASAS ADLMR dataset, BiGRU+Q2L outperforms state-of-the-art baselines, whereas Seq2Res demonstrates potential but its generation quality limits end-to-end gains. The results indicate that accurate resident separation substantially enhances recognition, motivating future work on improving sequence generation and applying post-processing to refine separated sequences.

Abstract

This paper presents two models to address the problem of multi-person activity recognition using ambient sensors in a home. The first model, Seq2Res, uses a sequence generation approach to separate sensor events from different residents. The second model, BiGRU+Q2L, uses a Query2Label multi-label classifier to predict multiple activities simultaneously. Performances of these models are compared to a state-of-the-art model in different experimental scenarios, using a state-of-the-art dataset of two residents in a home instrumented with ambient sensors. These results lead to a discussion on the advantages and drawbacks of resident separation and multi-label classification for multi-person activity recognition.

Figures (3)

• Figure 1: Framework of the proposed Seq2Res model for resident separation.
• Figure 2: Framework of the proposed BiGRU+Q2L model, with feature extraction from sensor events depicted on the left, and Query2Label transformer decoder for multi-label classification on the right.
• Figure 3: Example of an input sequence of Seq2Res model with its label and prediction. Each number represents the token of a sensor event.