Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Federated Learning for Cross-Modality Medical Image Segmentation via Augmentation-Driven Generalization

Sachin Dudda Nagaraju, Ashkan Moradi, Bendik Skarre Abrahamsen, Mattijs Elschot

TL;DR

This work considers a realistic FL scenario where each client holds single-modality data (CT or MRI), and systematically investigates augmentation strategies for cross-modality generalization, demonstrating strong cross-modality generalization without compromising data privacy.

Abstract

Artificial intelligence has emerged as a transformative tool in medical image analysis, yet developing robust and generalizable segmentation models remains difficult due to fragmented, privacy-constrained imaging data siloed across institutions. While federated learning (FL) enables collaborative model training without centralizing data, cross-modality domain shifts pose a critical challenge, particularly when models trained on one modality fail to generalize to another. Many existing solutions require paired multimodal data per patient or rely on complex architectures, both of which are impractical in real clinical settings. In this work, we consider a realistic FL scenario where each client holds single-modality data (CT or MRI), and systematically investigate augmentation strategies for cross-modality generalization. Using abdominal organ segmentation and whole-heart segmentation as representative multi-class and binary segmentation benchmarks, we evaluate convolution-based spatial augmentation, frequency-domain manipulation, domain-specific normalization, and global intensity nonlinear (GIN) augmentation. Our results show that GIN consistently outperforms alternatives in both centralized and federated settings by simulating cross-modality appearance variations while preserving anatomical structure. For the pancreas, Dice score improved from 0.073 to 0.437, a 498% gain. Our federated approach achieves 93-98% of centralized training accuracy, demonstrating strong cross-modality generalization without compromising data privacy, pointing toward feasible federated AI deployment across diverse healthcare systems.

Federated Learning for Cross-Modality Medical Image Segmentation via Augmentation-Driven Generalization

TL;DR

This work considers a realistic FL scenario where each client holds single-modality data (CT or MRI), and systematically investigates augmentation strategies for cross-modality generalization, demonstrating strong cross-modality generalization without compromising data privacy.

Abstract

Artificial intelligence has emerged as a transformative tool in medical image analysis, yet developing robust and generalizable segmentation models remains difficult due to fragmented, privacy-constrained imaging data siloed across institutions. While federated learning (FL) enables collaborative model training without centralizing data, cross-modality domain shifts pose a critical challenge, particularly when models trained on one modality fail to generalize to another. Many existing solutions require paired multimodal data per patient or rely on complex architectures, both of which are impractical in real clinical settings. In this work, we consider a realistic FL scenario where each client holds single-modality data (CT or MRI), and systematically investigate augmentation strategies for cross-modality generalization. Using abdominal organ segmentation and whole-heart segmentation as representative multi-class and binary segmentation benchmarks, we evaluate convolution-based spatial augmentation, frequency-domain manipulation, domain-specific normalization, and global intensity nonlinear (GIN) augmentation. Our results show that GIN consistently outperforms alternatives in both centralized and federated settings by simulating cross-modality appearance variations while preserving anatomical structure. For the pancreas, Dice score improved from 0.073 to 0.437, a 498% gain. Our federated approach achieves 93-98% of centralized training accuracy, demonstrating strong cross-modality generalization without compromising data privacy, pointing toward feasible federated AI deployment across diverse healthcare systems.
Paper Structure (32 sections, 5 equations, 3 figures, 5 tables, 1 algorithm)

This paper contains 32 sections, 5 equations, 3 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Federated Learning in Medical Imaging
  4. Domain Generalization for Cross-Modality Medical Image Segmentation
  5. Methodology
  6. Problem Formulation
  7. Global Intensity Non-linear Augmentation
  8. Design
  9. GIN Architecture
  10. Transformation Formulation
  11. Federated Training Framework
  12. Framework Overview
  13. Local Training with On-the-fly GIN Augmentation
  14. Global Model Aggregation
  15. Training Overview
  16. ...and 17 more sections

Figures (3)

  • Figure 1: Key challenges in federated multi-site medical image segmentation.
  • Figure 2: Overview of the proposed framework showing federated server coordination, local client training with GIN augmentation (Aug), and the detailed augmentation pipeline with random convolution and non-linear intensity transformations. The parameter $\alpha \in [0,1]$ controls the interpolation strength between original and augmented images, where $X_{\text{Aug}} = (1-\alpha)X + \alpha g(X)$.
  • Figure 3: Cross-modality medical image segmentation performance on MRI test data using CT–MRI training under different CT ratios. Results are shown for five organs (Spleen, Pancreas, Liver, Kidneys, and Gallbladder). The top row reports centralized training, while the bottom row reports federated training. Solid curves denote mean Dice scores across three runs, and shaded regions indicate $\pm$ one standard deviation. The dashed horizontal line indicates the MRI-only baseline.