Dynamic Allocation Hypernetwork with Adaptive Model Recalibration for Federated Continual Learning

TL;DR

The paper tackles the challenge of deploying federated continual learning (FCL) in medical settings where task streams are distinct, dynamic, and asynchronous. It proposes FedDAH, a server-side framework consisting of a Dynamic Allocation Hypernetwork (DAHyper) that maps task identities to full model weights to preserve knowledge, and Adaptive Model Recalibration (AMR) that calibrates continual optimization across time using task-level losses $L_{task}$ and $L_{R}$ with a similarity weight $W_s$, guided by Jensen–Shannon divergence. Experiments on the AMOS abdominal organ segmentation dataset with four clients show that FedDAH achieves superior Dice scores over baselines like FBL, FedWeIT, and FedSpace, validating both memory preservation and harmonious optimization across task streams. Ablation studies confirm the individual contributions of DAHyper, AMR, and similarity-based recalibration, and qualitative results illustrate more complete segmentations under FedDAH. This work advances practical deployment of FL in real-world medical scenarios by enabling robust server-side knowledge retention and coordinated optimization across heterogeneous, evolving task streams; code is available at the provided repository.

Abstract

Federated continual learning (FCL) offers an emerging pattern to facilitate the applicability of federated learning (FL) in real-world scenarios, where tasks evolve dynamically and asynchronously across clients, especially in medical scenario. Existing server-side FCL methods in nature domain construct a continually learnable server model by client aggregation on all-involved tasks. However, they are challenged by: (1) Catastrophic forgetting for previously learned tasks, leading to error accumulation in server model, making it difficult to sustain comprehensive knowledge across all tasks. (2) Biased optimization due to asynchronous tasks handled across different clients, leading to the collision of optimization targets of different clients at the same time steps. In this work, we take the first step to propose a novel server-side FCL pattern in medical domain, Dynamic Allocation Hypernetwork with adaptive model recalibration (FedDAH). It is to facilitate collaborative learning under the distinct and dynamic task streams across clients. To alleviate the catastrophic forgetting, we propose a dynamic allocation hypernetwork (DAHyper) where a continually updated hypernetwork is designed to manage the mapping between task identities and their associated model parameters, enabling the dynamic allocation of the model across clients. For the biased optimization, we introduce a novel adaptive model recalibration (AMR) to incorporate the candidate changes of historical models into current server updates, and assign weights to identical tasks across different time steps based on the similarity for continual optimization. Extensive experiments on the AMOS dataset demonstrate the superiority of our FedDAH to other FCL methods on sites with different task streams. The code is available:https://github.com/jinlab-imvr/FedDAH.

Figures (4)

• Figure 1: Task: Since disease evolves and treatment options change, different clients require to continually evolves on different task orders (asynchronous) or add new tasks (dynamic). Challenge: The construction of a server-side FCL model is challenged by: 1) Catastrophic forgetting for previously learned tasks. 2) Biased optimization due to asynchronous tasks.
• Figure 2: The framework of FedDAH: (a) Dynamic allocation hypernetwork preserves the mappings (task identity to model weights) by the hypernetwork to avoid knowledge forgetting. (b) Adaptive model recalibration assigns a calibration based on the contrastive similarity for continual optimization on asynchronous tasks.
• Figure 3: The visual results indicate the superior performance of FedDAH. Especially in the red box, we show a task optimized by other clients, and our FedDAH provides more complete segmentation.
• Figure 4: FedDAH ensures the continual optimization of different task streams in FCL. The horizontal axis is time step and vertical axis is dice score.