Efficient Deep Learning-based Estimation of the Vital Signs on Smartphones

TL;DR

This work tackles the need for continuous, on-device monitoring of vital signs by smartphone. It introduces MEDVSE, an end-to-end, fully convolutional approach that estimates heart rate, SpO2, and respiratory rate from fingertip video without pre-processing, achieving low parameter counts and real-time on-device inference. The authors propose multiple architectures (Base, FCN, Residual FCN, DCT, Modified-ConvNext) and demonstrate that the Residual FCN provides strong accuracy across datasets, while the DCT variant offers extreme efficiency. A public smartphone-derived dataset (MTHS) is released to support evaluation and future development, underscoring the study’s potential to enable scalable, privacy-preserving mobile health monitoring on commodity devices.

Abstract

With the increasing use of smartphones in our daily lives, these devices have become capable of performing many complex tasks. Concerning the need for continuous monitoring of vital signs, especially for the elderly or those with certain types of diseases, the development of algorithms that can estimate vital signs using smartphones has attracted researchers worldwide. In particular, researchers have been exploring ways to estimate vital signs, such as heart rate, oxygen saturation levels, and respiratory rate, using algorithms that can be run on smartphones. However, many of these algorithms require multiple pre-processing steps that might introduce some implementation overheads or require the design of a couple of hand-crafted stages to obtain an optimal result. To address this issue, this research proposes a novel end-to-end solution to mobile-based vital sign estimation using deep learning that eliminates the need for pre-processing. By using a fully convolutional architecture, the proposed model has much fewer parameters and less computational complexity compared to the architectures that use fully-connected layers as the prediction heads. This also reduces the risk of overfitting. Additionally, a public dataset for vital sign estimation, which includes 62 videos collected from 35 men and 27 women, is provided. Overall, the proposed end-to-end approach promises significantly improved efficiency and performance for on-device health monitoring on readily available consumer electronics.

Figures (8)

• Figure 1: The proposed architectures for vital sign estimation, there are five different architectures provided. The Base model represents the commonly used architecture for this task, which is used as a baseline. The guide section provides detailed information about the blocks in the proposed architectures.
• Figure 2: The setup for data collection.
• Figure 3: Heart rate and SpO2 distributions for the MTHS dataset.
• Figure 4: A plot of ground truth heart rate and its estimation by the best performing model (Residual FCN) on BIDMC dataset.
• Figure 5: A plot of ground truth SpO2 and its estimation by the best performing model on the BIDMC dataset.