Towards High-Quality Image Segmentation: Improving Topology Accuracy by Penalizing Neighbor Pixels

Abstract

Standard deep learning models for image segmentation cannot guarantee topology accuracy, failing to preserve the correct number of connected components or structures. This, in turn, affects the quality of the segmentations and compromises the reliability of the subsequent quantification analyses. Previous works have proposed to enhance topology accuracy with specialized frameworks, architectures, and loss functions. However, these methods are often cumbersome to integrate into existing training pipelines, they are computationally very expensive, or they are restricted to structures with tubular morphology. We present SCNP, an efficient method that improves topology accuracy by penalizing the logits with their poorest-classified neighbor, forcing the model to improve the prediction at the pixels' neighbors before allowing it to improve the pixels themselves. We show the effectiveness of SCNP across 13 datasets, covering different structure morphologies and image modalities, and integrate it into three frameworks for semantic and instance segmentation. Additionally, we show that SCNP can be integrated into several loss functions, making them improve topology accuracy. Our code can be found at https://jmlipman.github.io/SCNP-SameClassNeighborPenalization.

Figures (11)

• Figure 1: SCNP improves topology accuracy. Arrows indicate breakage in the structures.
• Figure 2: Logits (left) and logits after applying SCNP (right). SCNP propagates small values across the foreground-class logits, and large values across the background-class logits.
• Figure 3: Logits after applying SCNP and softmax normalization.
• Figure 4: $\beta_{0e}$ obtained by Cross Entropy Dice loss after optimizing the standard logits (CEDice), the SCNP-penalized logits ($\overline{CEDice}$), both (CEDice+$\overline{CEDice}$), and the top two loss functions across the 13 datasets. Datasets are grouped into four categories (① ② ③ ④) based on their image type (medical, non-medical) and structures' morphologies (tubular, non-tubular, rounded). First panel: Number of datasets in which each loss function led the most accurate segmentations topologically.
• Figure 5: Segmentations achieved by the baseline Cross Entropy Dice loss optimizing the standard logits (CEDice), the SCNP-penalized logits ($\overline{CEDice}$), and Tversky loss (the best-performing loss after CEDice) in eight datasets (two from each dataset group). Arrows indicate topological errors.