Federated Continual 3D Segmentation With Single-round Communication

TL;DR

This work tackles federated continual learning (FCL) for multi-class 3D segmentation in settings where new clients join and label spaces evolve. It proposes Multi-Model Distillation in the Server (MMDS), a one-time aggregation strategy that uses public unlabeled data and entropy-based pseudo-labeling to distill knowledge from independently trained client models into a global model, thereby removing the need for frequent synchronized communication. MMDS enables heterogeneous client architectures, supports incremental client updates, and substantially reduces communication overhead compared to traditional per-round aggregation while maintaining competitive performance, as demonstrated on six public 3D abdominal CT datasets. The results show robust generalization and personalization capabilities, with limitations arising from distillation data limitations and domain gaps, guiding future work toward data-free distillation and domain-adaptive strategies.

Abstract

Federated learning seeks to foster collaboration among distributed clients while preserving the privacy of their local data. Traditionally, federated learning methods assume a fixed setting in which client data and learning objectives remain constant. However, in real-world scenarios, new clients may join, and existing clients may expand the segmentation label set as task requirements evolve. In such a dynamic federated analysis setup, the conventional federated communication strategy of model aggregation per communication round is suboptimal. As new clients join, this strategy requires retraining, linearly increasing communication and computation overhead. It also imposes requirements for synchronized communication, which is difficult to achieve among distributed clients. In this paper, we propose a federated continual learning strategy that employs a one-time model aggregation at the server through multi-model distillation. This approach builds and updates the global model while eliminating the need for frequent server communication. When integrating new data streams or onboarding new clients, this approach efficiently reuses previous client models, avoiding the need to retrain the global model across the entire federation. By minimizing communication load and bypassing the need to put unchanged clients online, our approach relaxes synchronization requirements among clients, providing an efficient and scalable federated analysis framework suited for real-world applications. Using multi-class 3D abdominal CT segmentation as an application task, we demonstrate the effectiveness of the proposed approach.

Figures (8)

• Figure 1: The pipeline for the FCL task. In an FCL setting, new clients may join the federation over time, and the data distribution of existing clients can shift as additional data becomes available locally.
• Figure 2: Proposed MMDS framework. Initially, each client trains a model locally and uploads it to the central server. The server aggregates the local models using knowledge distillation on publicly available data to produce a global model, which is redistributed to clients. Each client integrates the global model with its local model for inference, preserving its knowledge while benefiting from shared knowledge. When new clients join or existing clients evolve, the new or revised local models are uploaded to the server. The server then performs distillation again to incorporate the new contributions into the global model, with the updated global model redistributed to all clients. This process is repeated when updates occur on the client side.
• Figure 3: Comparison of communication and computation loads between the classic MAPCR-type method FedAvg and the proposed method during 5-stage FCL segmentation across five clients. Here $1 \times$ represents the computation required to train a single model to convergence (assumed to be similar across client models).
• Figure 5: Class-wise DICE score performance of each client before and after the 5-stage FCL. FCL allows clients to segment classes that are not labeled in their local datasets by sharing knowledge with other clients.
• Figure 6: DICE score performance of the global model after 5-stage FCL with homogeneous and heterogeneous model architecture.