Accurate Fine-Grained Segmentation of Human Anatomy in Radiographs via Volumetric Pseudo-Labeling

TL;DR

A novel method for extracting fine-grained anatomical structures in CXR using pseudo-labeling of three-dimensional computed tomography (CT) scans paired with CT projection offers a promising approach for detailed anatomical segmentation of CxR with a high agreement with human annotators.

Abstract

Purpose: Interpreting chest radiographs (CXR) remains challenging due to the ambiguity of overlapping structures such as the lungs, heart, and bones. To address this issue, we propose a novel method for extracting fine-grained anatomical structures in CXR using pseudo-labeling of three-dimensional computed tomography (CT) scans. Methods: We created a large-scale dataset of 10,021 thoracic CTs with 157 labels and applied an ensemble of 3D anatomy segmentation models to extract anatomical pseudo-labels. These labels were projected onto a two-dimensional plane, similar to the CXR, allowing the training of detailed semantic segmentation models for CXR without any manual annotation effort. Results: Our resulting segmentation models demonstrated remarkable performance on CXR, with a high average model-annotator agreement between two radiologists with mIoU scores of 0.93 and 0.85 for frontal and lateral anatomy, while inter-annotator agreement remained at 0.95 and 0.83 mIoU. Our anatomical segmentations allowed for the accurate extraction of relevant explainable medical features such as the cardio-thoracic-ratio. Conclusion: Our method of volumetric pseudo-labeling paired with CT projection offers a promising approach for detailed anatomical segmentation of CXR with a high agreement with human annotators. This technique may have important clinical implications, particularly in the analysis of various thoracic pathologies.

Figures (10)

• Figure 1: A flowchart describing the PAX-Ray++ dataset generation process. (a) We collect publicly available datasets containing different anatomical structures. (b) We train an ensemble of nnUNets of each expert domain and infer them on a shared dataset. (c) We merge the 3D predictions, apply anatomical priors and retrain. (d) We infer the final nnUNet on 10K chest CTs. (e) We apply a CT and label projection to generate a chest X-Ray dataset which we apply anatomical-prior-based postprocessing to collect the final dataset.
• Figure 2: a) We show qualitative results for our CXAS model on frontal projections. We highlight masks for the respiratory system, bones, vasculatory system and abdomen. a) We show qualitative results for our CXAS model on lateral projections. We highlight masks for the respiratory system, bones, vasculatory system and abdomen. c) We show the performance of our CXAS model in terms of mIoU, mDICE and mHausdorff distance for frontal (top row) and lateral (bottom row) images.
• Figure 3: a) We show qualitative results for our CXAS model on frontal CXR as well as two expert manual annotations. We highlight masks for the respiratory system, bones, vasculatory system and abdomen. b) We show the performance of our CXAS model in terms of mIoU, mDICE and mHausdorff distance for frontal (top row) and lateral (bottom row) images.
• Figure 4: a) qualitative examples of the calculation of the CTR for presence/absence of cardiomegaly. b) We show the CTR distributions of the PA CXR of the MIMIC dataset for sex, pathology and age-group. We further show the predictive value of the CTR for cardiomegaly. c) We show qualitative examples of the calculation of the SCD for presence/absence of scoliosis. d) We show the SCD distributions of the PA CXR of the MIMIC dataset for sex, pathology and age-group. We further show the predictive value of the SCD for scoliosis.
• Figure 5: Comparison of different real and projected samples for the frontal and lateral view