Integrating Background Knowledge in Medical Semantic Segmentation with Logic Tensor Networks

TL;DR

The paper tackles data scarcity in medical image segmentation by injecting domain knowledge as soft logical constraints within a SwinUNETR-based segmentation framework using Logic Tensor Networks. This neurosymbolic approach yields higher Dice scores, particularly in low-data regimes, and improves anatomical plausibility through constraints like connectedness, nesting, and volume similarity. The findings demonstrate that background knowledge can regularize learning and guide segmentation toward clinically meaningful outputs without extra labeled data. The work suggests broader applicability of neurosymbolic conditioning to other medical segmentation tasks and highlights directions for stronger constraint enforcement and automated knowledge discovery.

Abstract

Semantic segmentation is a fundamental task in medical image analysis, aiding medical decision-making by helping radiologists distinguish objects in an image. Research in this field has been driven by deep learning applications, which have the potential to scale these systems even in the presence of noise and artifacts. However, these systems are not yet perfected. We argue that performance can be improved by incorporating common medical knowledge into the segmentation model's loss function. To this end, we introduce Logic Tensor Networks (LTNs) to encode medical background knowledge using first-order logic (FOL) rules. The encoded rules span from constraints on the shape of the produced segmentation, to relationships between different segmented areas. We apply LTNs in an end-to-end framework with a SwinUNETR for semantic segmentation. We evaluate our method on the task of segmenting the hippocampus in brain MRI scans. Our experiments show that LTNs improve the baseline segmentation performance, especially when training data is scarce. Despite being in its preliminary stages, we argue that neurosymbolic methods are general enough to be adapted and applied to other medical semantic segmentation tasks.

Figures (3)

• Figure 1: Ground truth data. The yellow and green areas are clearly distinct.
• Figure 2: Predicted data. The green border around the yellow area is implausible.
• Figure 4: Dice coefficient example calculation. Here, Dice = $\frac{2 \cdot 4}{6+6} \simeq 0.666$.