FeTTL: Federated Template and Task Learning for Multi-Institutional Medical Imaging

TL;DR

The paper tackles distribution shifts in multi-institution medical imaging under privacy constraints. It introduces FeTTL, a federated framework that jointly learns a global image template and a downstream task model via a federated whitening–coloring harmonization, enabling cross-site data harmonization without sharing raw data. FeTTL proceeds in three stages—federated harmonization, initial task learning, and joint template and task learning—and demonstrates significant improvements on retinal fundus disc segmentation across five sites and CAMELYON16 metastases classification over strong FL baselines. The results support FeTTL as a practical, scalable approach for privacy-preserving, robust multi-institutional medical imaging AI.

Abstract

Federated learning enables collaborative model training across geographically distributed medical centers while preserving data privacy. However, domain shifts and heterogeneity in data often lead to a degradation in model performance. Medical imaging applications are particularly affected by variations in acquisition protocols, scanner types, and patient populations. To address these issues, we introduce Federated Template and Task Learning (FeTTL), a novel framework designed to harmonize multi-institutional medical imaging data in federated environments. FeTTL learns a global template together with a task model to align data distributions among clients. We evaluated FeTTL on two challenging and diverse multi-institutional medical imaging tasks: retinal fundus optical disc segmentation and histopathological metastasis classification. Experimental results show that FeTTL significantly outperforms the state-of-the-art federated learning baselines (p-values <0.002) for optical disc segmentation and classification of metastases from multi-institutional data. Our experiments further highlight the importance of jointly learning the template and the task. These findings suggest that FeTTL offers a principled and extensible solution for mitigating distribution shifts in federated learning, supporting robust model deployment in real-world, multi-institutional environments.

Figures (4)

• Figure 1: Federated Template and Task Learning (FeTTL) framework showing the major steps of a) federated harmonization process, b) initial task learning, and c) the joint template and task learning. The arrows show the flow of information between the client and the server. Global model data is sent to each client shown in black arrows, while the colored arrows represent local data collection.
• Figure 2: Qualitative performance of FeTTL versus FedBN, FedProx, FedDG and FedHarmony for retina disc segmentation. The violin plots show the distribution of test Dice scores and the box plot on the $y$-axis for a test site on the $x$-axis. The box blot shows the median and the inter quartile range.
• Figure 3: Clustering performance of FeTTL on retinal images using t-SNE tsne illustrates the representation of the test image on HSV space for the original, the FeTTL-local and FeTTL-global and the templates obtained, respectively. Each color represents a different site. The cluster of harmonized images appear to be more compact for FeTTL-global over FeTTL-local.
• Figure 4: Impact of the template initialization on FeTTL for retinal disc segmentation. The figure shows whisker plots of $\mathcal{M}_{\theta,k}$ for different $\mathcal{T}_{\text{global}}$ initializations on the test data: (1) an image from site $k$, $\mathcal{I}_k$; (2) a template from $k$, such that $\mathcal{T}_k= enc(I_k)$; or (3) a random $\mathcal{T}_{\text{global}} \sim \mathcal{N}(0,1)$. The best performance on Dice scores was achieved for the FeTTL using a template from $k$, such that $\mathcal{T}_k= enc(I_k)$.