Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Longitudinal Segmentation of MS Lesions via Temporal Difference Weighting

Maximilian Rokuss, Yannick Kirchhoff, Saikat Roy, Balint Kovacs, Constantin Ulrich, Tassilo Wald, Maximilian Zenk, Stefan Denner, Fabian Isensee, Philipp Vollmuth, Jens Kleesiek, Klaus Maier-Hein

TL;DR

This work introduces a novel approach that explicitly incorporates temporal differences between baseline and follow-up scans through a unique architectural inductive bias called Difference Weighting Block, which merges features from two timepoints, emphasizing changes between scans.

Abstract

Accurate segmentation of Multiple Sclerosis (MS) lesions in longitudinal MRI scans is crucial for monitoring disease progression and treatment efficacy. Although changes across time are taken into account when assessing images in clinical practice, most existing deep learning methods treat scans from different timepoints separately. Among studies utilizing longitudinal images, a simple channel-wise concatenation is the primary albeit suboptimal method employed to integrate timepoints. We introduce a novel approach that explicitly incorporates temporal differences between baseline and follow-up scans through a unique architectural inductive bias called Difference Weighting Block. It merges features from two timepoints, emphasizing changes between scans. We achieve superior scores in lesion segmentation (Dice Score, Hausdorff distance) as well as lesion detection (lesion-level $F_1$ score) as compared to state-of-the-art longitudinal and single timepoint models across two datasets. Our code is made publicly available at www.github.com/MIC-DKFZ/Longitudinal-Difference-Weighting.

Longitudinal Segmentation of MS Lesions via Temporal Difference Weighting

TL;DR

This work introduces a novel approach that explicitly incorporates temporal differences between baseline and follow-up scans through a unique architectural inductive bias called Difference Weighting Block, which merges features from two timepoints, emphasizing changes between scans.

Abstract

Accurate segmentation of Multiple Sclerosis (MS) lesions in longitudinal MRI scans is crucial for monitoring disease progression and treatment efficacy. Although changes across time are taken into account when assessing images in clinical practice, most existing deep learning methods treat scans from different timepoints separately. Among studies utilizing longitudinal images, a simple channel-wise concatenation is the primary albeit suboptimal method employed to integrate timepoints. We introduce a novel approach that explicitly incorporates temporal differences between baseline and follow-up scans through a unique architectural inductive bias called Difference Weighting Block. It merges features from two timepoints, emphasizing changes between scans. We achieve superior scores in lesion segmentation (Dice Score, Hausdorff distance) as well as lesion detection (lesion-level score) as compared to state-of-the-art longitudinal and single timepoint models across two datasets. Our code is made publicly available at www.github.com/MIC-DKFZ/Longitudinal-Difference-Weighting.
Paper Structure (14 sections, 2 equations, 3 figures, 4 tables)

This paper contains 14 sections, 2 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Integrating Prior Images for Enhanced Segmentation
  4. Difference Weighting Block
  5. Experiments
  6. Datasets
  7. Evaluation Metrics
  8. Implementation details
  9. Results and Discussion
  10. Conclusion
  11. Acknowledgments.
  12. Disclosure of Interests.
  13. Statistical Analysis
  14. Results without Size Filtering

Figures (3)

  • Figure 1: Left: Pipeline of our proposed model. First, the prior image (baseline) and current image (follow-up) are both encoded by a shared encoder. Subsequently, the Difference Weighting Block is applied on all skip connections to merge these encoded representations, emphasizing dissimilarities between the two images. This temporally enhanced representation is then fed to the decoder to produce the segmentation of the current image. Right: Architecture of the Difference Weighting Block. Firstly, the difference between the features extracted from the current and prior images is computed, followed by an Instance Norm. This result is then multiplied with the current image features and finally a residual connection is added.
  • Figure 2: Qualitative results on the Ljubljana dataset. Compared to related work, our proposed method demonstrates superior performance in volumetric delineation of MS lesions and successfully identifies lesions missed by other methods.
  • Figure 3: Distribution of Dice and lesion-based F1 scores on the Ljubljana test set. The longitudinal baselines are depicted in blue, nnUNet as the best single timepoint baseline in orange, and our proposed Difference Weighting approach in green. The figure depicts the superiority of our proposed method in terms of both the mean and the distribution, particularly evident in the F1 score. The narrower tails of the distribution indicate more precise segmentations, especially for challenging cases.