Two-Stream Thermal Imaging Fusion for Enhanced Time of Birth Detection in Neonatal Care

TL;DR

The paper addresses inaccuracies in manual Time of Birth (ToB) documentation in neonatal care by proposing a two-stream fusion framework that processes thermal imaging through a static image stream ($EfficientNet$) and a dynamic video stream ($MoViNet$) to produce a fusion score $p_{fusion}$. A score-aggregation module with a lightweight LSTM combines $p_{fusion}$ and an image-based score $p_{vnb}$ to estimate ToB as $\\\hat{T}_{birth}=\\arg\\max_t \\hat{y}_{joint}(t)$ with a threshold $\\gamma=0.95$, improving robustness to missed detections. The authors validate on a dataset of 611 thermal birth videos (and 258 image-based samples) from Stavanger University Hospital, using manual second-precision ToB annotations and VNB labels; evaluation on 35 test videos shows 100% ToB detection with a median ToB error of 2 seconds and mean error of 4.5 seconds, along with strong precision/recall and efficient runtime suitable for real-time NRAA timelines. This approach advances privacy-preserving birth identification via thermal imaging and provides a practical, scalable pathway toward automated, detailed resuscitation timelines (NRAA) in neonatal care.

Abstract

Around 10% of newborns require some help to initiate breathing, and 5\% need ventilation assistance. Accurate Time of Birth (ToB) documentation is essential for optimizing neonatal care, as timely interventions are vital for proper resuscitation. However, current clinical methods for recording ToB often rely on manual processes, which can be prone to inaccuracies. In this study, we present a novel two-stream fusion system that combines the power of image and video analysis to accurately detect the ToB from thermal recordings in the delivery room and operating theater. By integrating static and dynamic streams, our approach captures richer birth-related spatiotemporal features, leading to more robust and precise ToB estimation. We demonstrate that this synergy between data modalities enhances performance over single-stream approaches. Our system achieves 95.7% precision and 84.8% recall in detecting birth within short video clips. Additionally, with the help of a score aggregation module, it successfully identifies ToB in 100% of test cases, with a median absolute error of 2 seconds and an absolute mean deviation of 4.5 seconds compared to manual annotations.

Figures (2)

• Figure 1: Overview of our proposed two-stream system for Time of Birth detection. At each second, a sliding window containing the previous 25 frames (3 seconds) is utilized as input $x(t)$ to our model. The video clip is processed independently in each stream by pretrained backbones, extracting static and temporal features that are concatenated to produce richer probability scores $p_{fusion}$. These scores pass through a score aggregation module where they are combined with image-specific scores $p_{vnb}$ and processed by an auxiliary model using a moving window $w$ to refine ToB predictions.
• Figure 2: Examples of predicted event (green), transition (red), and joint (purple) signals around ToB. $T_{birth}$ is depicted in black, and $\hat{T}_{birth}$ in cyan. $a)$ Ground truth signals. $b)$ Event and transition signals correctly identify the ToB. $c)$ The auxiliary model avoids false positives by finding the best trade-off between event and transition signals. $d) \text{ and } e)$ If any signal fails, the auxiliary model relies on the other signal to estimate ToB, acting as a fallback system.