Thermal Imaging and Radar for Remote Sleep Monitoring of Breathing and Apnea

TL;DR

This paper tackles non-contact sleep monitoring by comparing thermal imaging and FMCW radar for breathing waveform extraction and apnea detection. It introduces a synchronized multimodal setup and demonstrates that thermal imaging outperforms radar in apnea detection while enabling a multimodal classification pathway to distinguish between obstructive and central sleep apnea. The study provides a dataset of 10 patients with ground truth PSG and open-source data and code, and shows that non-contact sensors can approach clinical utility for diagnosing sleep disorders in patients unable to tolerate contact-based monitoring. The findings suggest a path toward portable, non-invasive sleep diagnostics with potential broad impact on underdiagnosed populations.

Abstract

Polysomnography (PSG), the current gold standard method for monitoring and detecting sleep disorders, is cumbersome and costly. At-home testing solutions, known as home sleep apnea testing (HSAT), exist. However, they are contact-based, a feature which limits the ability of some patient populations to tolerate testing and discourages widespread deployment. Previous work on non-contact sleep monitoring for sleep apnea detection either estimates respiratory effort using radar or nasal airflow using a thermal camera, but has not compared the two or used them together. We conducted a study on 10 participants, ages 34 - 78, with suspected sleep disorders using a hardware setup with a synchronized radar and thermal camera. We show the first comparison of radar and thermal imaging for sleep monitoring, and find that our thermal imaging method outperforms radar significantly. Our thermal imaging method detects apneas with an accuracy of 0.99, a precision of 0.68, a recall of 0.74, an F1 score of 0.71, and an intra-class correlation of 0.70; our radar method detects apneas with an accuracy of 0.83, a precision of 0.13, a recall of 0.86, an F1 score of 0.22, and an intra-class correlation of 0.13. We also present a novel proposal for classifying obstructive and central sleep apnea by leveraging a multimodal setup. This method could be used accurately detect and classify apneas during sleep with non-contact sensors, thereby improving diagnostic capacities in patient populations unable to tolerate current technology.

Figures (6)

• Figure 1: Airflow and respiratory effort waveforms measured with sleep lab sensors depicting normal breathing, obstructive apnea, and central apnea. During the onset of CSA, we notice anomalies in both the airflow and the respiratory effort waveforms; these anomalies manifest as attenuating factors that lower the amplitude of the two waveforms. Unlike CSA, the occurrence of OSA can be detected only through the airflow, which experiences a similar reduction in amplitude as the previous case. In comparison, the respiratory effort does not experience the same degree of attenuation for OSA.
• Figure 2: Proposed pipeline for breathing detection from the radar and thermal modalities, followed by subsequent apnea detection. First, we crop the frames acquired from the thermal camera (top) to focus only on the region near the nose. Then, we perform a global spatial averaging operation to collapse the video into a single time-series sequence, which, when filtered, gives us the breathing waveforms. For the radar (middle), we perform a standard Range-Doppler analysis to identify the approximate location of the patient, i.e., within a window of range bins. Once identified, we take an SNR-based weighted average of the extracted range bins to obtain breathing waveforms, which can then be filtered to increase waveform quality. Finally, we employ an envelope detection algorithm (bottom) to extract the upper and lower envelope, which can then be analyzed to detect anomalous regions on the waveform, i.e., regions with apnea.
• Figure 3: Our experimental hardware setup consisting of a thermal camera, radar, and data-processing auxiliaries located in a sleep lab. In this particular setting, we have placed the thermal camera and the radar to the right of the bed. Our setup also includes a microcontroller that is connected to the existing in-lab PSG hardware. This microcontroller sends pulse trains that can be used to synchronize the ground truth annotations with the captured recordings.
• Figure 4: Bland-Altman plot for breathing rates estimated by our radar (left) and thermal (right) modalities.
• Figure 5: Our apnea confidence scores and breathing waveforms are estimated from our thermal and radar recordings for several apnea events.