Federated-Continual Dynamic Segmentation of Histopathology guided by Barlow Continuity

TL;DR

This work addresses the dual challenge of data shifts in privacy-preserving histopathology learning by introducing Dynamic Barlow Continuity (DynBC), which evaluates and filters model updates on a public reference dataset to enforce spatio-temporal continuity. By connecting a Barlow Twins–style redundancy-reduction objective to update aggregation, DynBC guides the global model toward shift-invariant representations in both Federated and Continual Learning settings. Empirical results on BCSS and Semicol demonstrate that DynBC substantially improves Dice scores under both Client Drift and Catastrophic Forgetting, and remains competitive when these challenges occur simultaneously, all while preserving data privacy. The approach offers a practical route to robust, dynamic learning in histopathology that does not rely on access to old data or task labels, with broad implications for privacy-aware AI deployment in medical imaging.

Abstract

Federated- and Continual Learning have been established as approaches to enable privacy-aware learning on continuously changing data, as required for deploying AI systems in histopathology images. However, data shifts can occur in a dynamic world, spatially between institutions and temporally, due to changing data over time. This leads to two issues: Client Drift, where the central model degrades from aggregating data from clients trained on shifted data, and Catastrophic Forgetting, from temporal shifts such as changes in patient populations. Both tend to degrade the model's performance of previously seen data or spatially distributed training. Despite both problems arising from the same underlying problem of data shifts, existing research addresses them only individually. In this work, we introduce a method that can jointly alleviate Client Drift and Catastrophic Forgetting by using our proposed Dynamic Barlow Continuity that evaluates client updates on a public reference dataset and uses this to guide the training process to a spatially and temporally shift-invariant model. We evaluate our approach on the histopathology datasets BCSS and Semicol and prove our method to be highly effective by jointly improving the dice score as much as from 15.8% to 71.6% in Client Drift and from 42.5% to 62.8% in Catastrophic Forgetting. This enables Dynamic Learning by establishing spatio-temporal shift-invariance.

Figures (5)

• Figure 1: DynBC detects drastic changes of predictions from spatio-temporally trained models (left). It guides the training process to a more robust parameter representation by cancelling such model updates in spatio-temporal scenarios: In Federated Learning (spatial), it cancels model updates from certain clients, while in temporal Continual Learning (temporal) it rolls back to the previous model state, mitigating the problems through distribution shifts (red) such as Client Drift and Catastrophic Forgetting (right).
• Figure 2: DynBC (1) measures the shift invariance of a model update. A small DynBC indicates less biased to a certain distribution, as it shows shift invariance when updating. DynBC for Dynamic Learning (2): We compare each potential model update to the previous Continual Learning (CL) or global Federated Learning (FL) model. The resulting distance decides, whether the update is applied or ignored.
• Figure 3: Setting of the experiments for Client Drift, Catastrophic Forgetting and their combination
• Figure 4: Comparison of our method (DynBC) with Rehearsal and the baseline without any method on the datasets BCSS and Semicol in CD
• Figure 5: Comparison of our method (DynBC) with Rehearsal and the baseline without any method on the datasets BCSS and Semicol in CF