When to Stop Federated Learning: Zero-Shot Generation of Synthetic Validation Data with Generative AI for Early Stopping

TL;DR

This work tackles the inefficiency of fixed-round training in federated learning by introducing a zero-shot synthetic validation pipeline that enables early stopping. A central server generates a fixed synthetic validation set using generative AI (Stable Diffusion variants and RoentGen) and tracks performance during FL to identify near-optimal stopping rounds via a relative-improvement criterion. Across a chest X-ray multi-label task with $N=100$ devices and non-IID data, the method achieves substantial speedups (up to about $4.67\times$) while keeping accuracy within roughly $1\%$ of the optimum, and it remains effective across multiple FL algorithms. An ablation shows that a domain-tuned generator yields additional efficiency gains (about $8\%$) without harming accuracy, highlighting practical benefits for resource-constrained FL deployments.

Abstract

Federated Learning (FL) enables collaborative model training across decentralized devices while preserving data privacy. However, FL methods typically run for a predefined number of global rounds, often leading to unnecessary computation when optimal performance is reached earlier. In addition, training may continue even when the model fails to achieve meaningful performance. To address this inefficiency, we introduce a zero-shot synthetic validation framework that leverages generative AI to monitor model performance and determine early stopping points. Our approach adaptively stops training near the optimal round, thereby conserving computational resources and enabling rapid hyperparameter adjustments. Numerical results on multi-label chest X-ray classification demonstrate that our method reduces training rounds by up to 74% while maintaining accuracy within 1% of the optimal.

Figures (3)

• Figure 1: Illustration of our proposed synthetic validation-based early stopping approach in FL. The central server monitors validation accuracy on synthetic data (blue line) to determine the early stopping point ($\starletfill$) near the optimal round, achieving comparable performance to the actual optimal round ($\starletfill$) while avoiding wasteful computational rounds.
• Figure 2: Illustration of global loss trajectories and corresponding decision boundary patterns across different federated training scenarios. Note that each arrow denotes the aggregation of each global round in FL.
• Figure 3: Comparison of early stopping performance across six FL methods using different Stable Diffusion variants. Each subplot shows the test accuracy achieved by our synthetic validation approach (colored markers: $\trianglepbfill$, $\trianglepafill$, $\squadfill$, $\pentagofill$) compared to the optimal round performance ($\starletfill$). Note that we select the best-performing configuration.