Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FedPID: An Aggregation Method for Federated Learning

Leon Mächler, Gustav Grimberg, Ivan Ezhov, Manuel Nickel, Suprosanna Shit, David Naccache, Johannes C. Paetzold

TL;DR

Inspired by FedCostWAvg and FedPIDAvg, the authors' winning contributions to FETS21 and FETS2022, an improved aggregation strategy for federated and collaborative learning is proposed.

Abstract

This paper presents FedPID, our submission to the Federated Tumor Segmentation Challenge 2024 (FETS24). Inspired by FedCostWAvg and FedPIDAvg, our winning contributions to FETS21 and FETS2022, we propose an improved aggregation strategy for federated and collaborative learning. FedCostWAvg is a method that averages results by considering both the number of training samples in each group and how much the cost function decreased in the last round of training. This is similar to how the derivative part of a PID controller works. In FedPIDAvg, we also included the integral part that was missing. Another challenge we faced were vastly differing dataset sizes at each center. We solved this by assuming the sizes follow a Poisson distribution and adjusting the training iterations for each center accordingly. Essentially, this part of the method controls that outliers that require too much training time are less frequently used. Based on these contributions we now adapted FedPIDAvg by changing how the integral part is computed. Instead of integrating the loss function we measure the global drop in cost since the first round.

FedPID: An Aggregation Method for Federated Learning

TL;DR

Inspired by FedCostWAvg and FedPIDAvg, the authors' winning contributions to FETS21 and FETS2022, an improved aggregation strategy for federated and collaborative learning is proposed.

Abstract

This paper presents FedPID, our submission to the Federated Tumor Segmentation Challenge 2024 (FETS24). Inspired by FedCostWAvg and FedPIDAvg, our winning contributions to FETS21 and FETS2022, we propose an improved aggregation strategy for federated and collaborative learning. FedCostWAvg is a method that averages results by considering both the number of training samples in each group and how much the cost function decreased in the last round of training. This is similar to how the derivative part of a PID controller works. In FedPIDAvg, we also included the integral part that was missing. Another challenge we faced were vastly differing dataset sizes at each center. We solved this by assuming the sizes follow a Poisson distribution and adjusting the training iterations for each center accordingly. Essentially, this part of the method controls that outliers that require too much training time are less frequently used. Based on these contributions we now adapted FedPIDAvg by changing how the integral part is computed. Instead of integrating the loss function we measure the global drop in cost since the first round.

Paper Structure

This paper contains 12 sections, 12 equations, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. FETS challenge
  3. Prior work
  4. Federated Averaging (FedAvg)
  5. Federated Cost Weighted Averaging (FedCostWAvg)
  6. Federated PID Weighted Averaging (FedPIDAvg)
  7. Methodology
  8. Improved Federated PID Weighted Averaging (FedPID)
  9. U-Net for Brain Tumor Segmentation
  10. Poisson-distribution modeling of the data samples per center
  11. Results
  12. Conclusion

Figures (2)

  • Figure 1: The schematic illustration shows the concept of federated learning. It depicts multiple data centers that form a federation. Each data center stores its training data of different sizes locally and trains the same model for a specific task, such as brain tumor segmentation in our case. During the aggregation step, the locally trained model weights are sent to a central server which then performs model aggregation and broadcasts the updated model back to the local centers. This process is repeated until the model converges or another stopping criterion is met.
  • Figure 2: The common 3D U-net architecture, which is used in many medical imaging tasks. It was provided as such by the FETS challenge; modifications were not allowed ronneberger2015upati2021federated.