Anatomically aware dual-hop learning for pulmonary embolism detection in CT pulmonary angiograms

TL;DR

This work tackles rapid and accurate PE detection in CT pulmonary angiograms by introducing three synergistic components: anatomically aware cropping to focus analysis on lung and heart regions, anatomically aware pretraining on 20 landmarks to imbue the model with body-structure knowledge, and a dual-hop detector that refines predictions across two processing passes. The approach is evaluated on the RSNA RSPECT dataset, with ablations showing additive improvements from cropping, pretraining, and the two-hop architecture, culminating in state-of-the-art performance (F1 ≈ 91.0% and AUC-PR ≈ 66.21) that rivals radiologist performance. The framework emphasizes efficiency and explainability, as evidenced by ROI-based cropping reducing input volume, Cutout augmentation improving regularization, and Integrated Gradients illustrating PE-centric reasoning. Overall, the three-stage, anatomically grounded design demonstrates strong generalization potential for medical imaging tasks beyond PE detection.

Abstract

Pulmonary Embolisms (PE) represent a leading cause of cardiovascular death. While medical imaging, through computed tomographic pulmonary angiography (CTPA), represents the gold standard for PE diagnosis, it is still susceptible to misdiagnosis or significant diagnosis delays, which may be fatal for critical cases. Despite the recently demonstrated power of deep learning to bring a significant boost in performance in a wide range of medical imaging tasks, there are still very few published researches on automatic pulmonary embolism detection. Herein we introduce a deep learning based approach, which efficiently combines computer vision and deep neural networks for pulmonary embolism detection in CTPA. Our method features novel improvements along three orthogonal axes: 1) automatic detection of anatomical structures; 2) anatomical aware pretraining, and 3) a dual-hop deep neural net for PE detection. We obtain state-of-the-art results on the publicly available multicenter large-scale RSNA dataset.

Figures (12)

• Figure 1: Top: Raw Input central Slice, with the lungs and heart segmentations highlighted in blue, and red, respectively; Bottom: Crop of the central slice. Images are scaled to the same size to better understand intuitively the increase in resolution brought by the cropping mechanism. Red borders capture the same anatomical structure. Note that in the cropped image, at a higher resolution, we can observe more structural details that can help with the diagnosis. The segmentation masks used to generate the crop are detected automatically using machine learning models described in Appendix \ref{['appendix:organ']}.
• Figure 2: Proposed workflow: each stage represents one of the contributions. Left: Anatomically Aware Segmentation and Cropping, through which data is specialised for PE detection. Middle: Anatomically Aware Pretraining on the related task of Anatomical Landmark Detection, through which the model is primed for our task of Pulmonary Embolism detection. Right Hopped training, through which model predictions are refined, over two hops of neural processing.
• Figure 3: Top: Raw Input central Slice, with the lungs and heart segmentations highlighted in blue, and red, respectively; Bottom: Crop of the central slice. Images are scaled to the same size to better understand intuitively the increase in resolution brought by the cropping mechanism. Red borders capture the same anatomical structure. Note that in the cropped image, at a higher resolution, we can observe more structural details that can help with the diagnosis. The segmentation masks used to generate the crop are detected automatically using machine learning models described in Appendix \ref{['appendix:organ']}.
• Figure 4: Visualization of a few anatomical landmarks used for our anatomical aware pretranining. On the left we visualize a traversal and a coronal sections of a body 3D-CT scan. On the right we highlight the different anatomical structures and mark the individual landmark points: (1) bronchial bifurcation, (2) bifurcation of left subclavian artery, (3) bifurcation of left common carotid artery and left subclavian artery, (4) bifurcation of left common carotid artery and brachiocephalic artery, (5) center of right kidney, (6) center of left kidney. Courtesy of Florin Ghesu.
• Figure 5: Hopped training, illustrated for $Hop_1$ and $Hop_2$. The model from $Hop_{1}$ receives as input only the 3D slice volume $X_{input}$, while the following $Hop_{2}$ receives an additional $psi_{1}$, corresponding to the aggregated features from $Hop_{1}$.