Beyond Fixed Rounds: Data-Free Early Stopping for Practical Federated Learning

TL;DR

The paper addresses hyperparameter tuning in privacy-preserving federated learning for medical imaging, where relying on validation data and fixed global rounds is costly. It introduces a data-free early stopping framework that monitors the growth of the global task vector v_r and its distance d_r to determine when to halt training, using a threshold tau and patience rho applied to the growth rate g_r. The approach is demonstrated across 10 FL methods on skin lesion and blood cell classification, achieving validation-level performance while reducing wasted computation and showing robustness to non-IID data, with average overhead of only about 9.5 extra rounds for skin lesion and 8.5 for blood cell. This data-free stopping technique offers a practical, privacy-preserving mechanism for hyperparameter tuning in decentralized medical imaging scenarios, enabling resource-efficient deployment of FL.

Abstract

Federated Learning (FL) facilitates decentralized collaborative learning without transmitting raw data. However, reliance on fixed global rounds or validation data for hyperparameter tuning hinders practical deployment by incurring high computational costs and privacy risks. To address this, we propose a data-free early stopping framework that determines the optimal stopping point by monitoring the task vector's growth rate using solely server-side parameters. The numerical results on skin lesion/blood cell classification demonstrate that our approach is comparable to validation-based early stopping across various state-of-the-art FL methods. In particular, the proposed framework spends an average of 47/20 (skin lesion/blood cell) rounds to achieve over 12.5%/10.3% higher performance than early stopping based on validation data. To the best of our knowledge, this is the first work to propose an early stopping framework for FL methods without using any validation data.

Figures (4)

• Figure 1: Illustration of resource inefficiency in FL hyperparameter tuning. Since standard protocols use a fixed number of global rounds, 'bad' configurations waste computational and communication resources.
• Figure 2: Illustration of the proposed data-free early stopping framework. The server monitors the growth rate of the task vector using only global model parameters and stops training once the growth rate falls below the threshold. Here, the color intensity of the task vector reflects its increasing magnitude, and vice versa.
• Figure 3: Test accuracy (%) and growth rate trajectories over global rounds with $\tau=0.01$ and $\rho=10$ for both proposed and validation-based early stopping. The zoom-in boxes highlight the regions around the early stopping points.
• Figure 4: Test accuracy (%) of FL methods evaluated under various threshold values with $\rho=10$ for both validation-based and proposed early stopping. The solid curves denote the mean test accuracy, while the shaded regions indicate the standard deviation.