OMuSense-23: A Multimodal Dataset for Contactless Breathing Pattern Recognition and Biometric Analysis

TL;DR

OMuSense-23 tackles the scarcity of multimodal non-contact biometric datasets by introducing a publicly available resource that combines mmWave radar and RGB-D camera data from 50 participants across three poses and four breathing activities. The paper details a full data collection and processing pipeline, including signal extraction (rPPG, chest displacement, BW, HW), feature engineering (statistical, fractal, entropy, HRV), and a suite of regression/classification benchmarks using both unimodal and fused features under multiple evaluation protocols. Key contributions include a comprehensive baseline analysis for pose and breathing-pattern recognition, structured protocols for LOSO, k-fold, and train/validation/test evaluations, and a first-of-its-kind dataset that balances gender representation and emergency-like breathing scenarios. The results demonstrate strong pose and breathing-pattern discrimination with multimodal fusion (up to ~0.87 pose accuracy and ~0.83 breathing accuracy) while biometric parameter estimation (age/height/weight) remains challenging, highlighting opportunities for future fusion and learning strategies in non-contact biometrics.

Abstract

In the domain of non-contact biometrics and human activity recognition, the lack of a versatile, multimodal dataset poses a significant bottleneck. To address this, we introduce the Oulu Multi Sensing (OMuSense-23) dataset that includes biosignals obtained from a mmWave radar, and an RGB-D camera. The dataset features data from 50 individuals in three distinct poses -- standing, sitting, and lying down -- each featuring four specific breathing pattern activities: regular breathing, reading, guided breathing, and apnea, encompassing both typical situations (e.g., sitting with normal breathing) and critical conditions (e.g., lying down without breathing). In our work, we present a detailed overview of the OMuSense-23 dataset, detailing the data acquisition protocol, describing the process for each participant. In addition, we provide, a baseline evaluation of several data analysis tasks related to biometrics, breathing pattern recognition and pose identification. Our results achieve a pose identification accuracy of 87\% and breathing pattern activity recognition of 83\% using features extracted from biosignals. The OMuSense-23 dataset is publicly available as resource for other researchers and practitioners in the field.

Figures (11)

• Figure 1: System setup containing an RGB-D Camera Intel Realsense D435 and an mmWave Radar Texas Instruments IWR1443 in a height configurable tripod.
• Figure 2: RGB-D Camera stream: RGB data (left) and depth information (right).
• Figure 3: Three principal waveform are obtained by the mmWave radar in its vital signs configuration: chest displacement (black), breathing waveform (blue) and heartbeat waveform (red).
• Figure 4: The three poses: Pose A, standing. Pose B, sitting. Pose C, lying down. Each of the poses stands within a range of 1.3 meters to the mmWave radar and RGB-D Camera.
• Figure 5: Experimental setup for bio-movement radar analysis: Pose A (left) shows the subject standing 70 cm from the radar for breathing tasks. In Pose B (center), the subject is seated, with torso movement restricted by the chair, altering Doppler shifts. Pose C (right) has the subject lying down, executing the same tasks without torso constraints. The red square depicts the real image captured by the camera.