MEDPSeg: Hierarchical polymorphic multitask learning for the segmentation of ground-glass opacities, consolidation, and pulmonary structures on computed tomography

TL;DR

This work proposes MEDPSeg, a hierarchical polymorphic multitask learning system that simultaneously performs segmentation of the lung parenchyma, airways, pulmonary artery, and lung lesions, all in a single forward prediction, with performance comparable to state-of-the-art methods specialized in each of those targets.

Abstract

The COVID-19 pandemic response highlighted the potential of deep learning methods in facilitating the diagnosis, prognosis and understanding of lung diseases through automated segmentation of pulmonary structures and lesions in chest computed tomography (CT). Automated separation of lung lesion into ground-glass opacity (GGO) and consolidation is hindered due to the labor-intensive and subjective nature of this task, resulting in scarce availability of ground truth for supervised learning. To tackle this problem, we propose MEDPSeg. MEDPSeg learns from heterogeneous chest CT targets through hierarchical polymorphic multitask learning (HPML). HPML explores the hierarchical nature of GGO and consolidation, lung lesions, and the lungs, with further benefits achieved through multitasking airway and pulmonary artery segmentation. Over 6000 volumetric CT scans from different partially labeled sources were used for training and testing. Experiments show PML enabling new state-of-the-art performance for GGO and consolidation segmentation tasks. In addition, MEDPSeg simultaneously performs segmentation of the lung parenchyma, airways, pulmonary artery, and lung lesions, all in a single forward prediction, with performance comparable to state-of-the-art methods specialized in each of those targets. Finally, we provide an open-source implementation with a graphical user interface at https://github.com/MICLab-Unicamp/medpseg.

Figures (10)

• Figure 1: Illustration of all target formats used in this work. Axial slices show the hierarchical properties of lung lesion annotation with whole lung in yellow, healthy tissue in red, lesions in magenta, GGO in green, and consolidated lung in blue. The airways and the pulmonary arterial tree are displayed in cyan and orange, respectively. All visualizations created with ITK-Snap yushkevich2006user.
• Figure 2: (A) Illustration of MEDPSeg architecture from initial feature extraction with the EfficientNet backbone, BiFPN with spatial attention (B) feature decoding, and the multitask (C) and polymorphic (D) head outputs. Multiple output formats are built on the fly with sum reductions and the applicable format is applied depending on the target format of the input image.
• Figure 3: During validation and for inference, MEDPSeg processes whole volumes through stacking axial predictions.
• Figure 4: Hierarchical Polymorphic Learning takes advantage of the hierarchical natures of annotations of lungs and pathologies. Airway and pulmonary artery labels are multitasked in separate outputs due to not being hierarchically consistent with the pathology hierarchy tree.
• Figure 5: Illustration of Poly AttUNet (P-AttUNet). A simpler UNet-like approach to HPML with one encoder and three decoders.