Compositional Segmentation of Cardiac Images Leveraging Metadata

TL;DR

This work proposes a novel multitask compositional segmentation approach that can simultaneously localize the heart in a cardiac image and perform part-based segmentation of different regions of interest, and demonstrates that this compositional approach achieves better results than direct segmentation of the anatomies.

Abstract

Cardiac image segmentation is essential for automated cardiac function assessment and monitoring of changes in cardiac structures over time. Inspired by coarse-to-fine approaches in image analysis, we propose a novel multitask compositional segmentation approach that can simultaneously localize the heart in a cardiac image and perform part-based segmentation of different regions of interest. We demonstrate that this compositional approach achieves better results than direct segmentation of the anatomies. Further, we propose a novel Cross-Modal Feature Integration (CMFI) module to leverage the metadata related to cardiac imaging collected during image acquisition. We perform experiments on two different modalities, MRI and ultrasound, using public datasets, Multi-disease, Multi-View, and Multi-Centre (M&Ms-2) and Multi-structure Ultrasound Segmentation (CAMUS) data, to showcase the efficiency of the proposed compositional segmentation method and Cross-Modal Feature Integration module incorporating metadata within the proposed compositional segmentation network. The source code is available: https://github.com/kabbas570/CompSeg-MetaData.

Figures (5)

• Figure 1: Influence of metadata on image quality, appearance, and intensity patterns: Analysis of MRI (M&Ms-2 dataset) and Ultrasound (CAMUS dataset) scans across different acquisition parameters and patient-specific information, i.e, disease, sex, and age.
• Figure 2: Overview of the proposed pipeline. The network has five encoder-decoder stages; only three are shown here for simplicity. The image encoder extracts features from the image; the two decoders perform super and sub-segmentation. The metadata is learned via MLP, followed by the interaction of image and metadata features using the CMFI module.
• Figure 3: The proposed CMFI Module. Each block's subscripts, $\boldsymbol{I}$ and $\boldsymbol{M}$, represent the image and metadata features, respectively.
• Figure 4: Visual comparison of our compositional approach with (W/) and without (WO/) the super-segmentation, metadata utilization strategy using FiLMlemay2021benefits, proposed CMFI module, and other comparative networks using M&Ms-2 dataset. Please zoom in for details.
• Figure 5: Qualitative comparison of proposed compositional approach with (W/) and without (WO/) the super-segmentation and metadata utilization strategy using CMFI module using CAMUS data. Please zoom in for details.