ProFL: Performative Robust Optimal Federated Learning

TL;DR

This work tackles performative distribution shifts in federated learning by targeting the performative optimal point (PO) under data contamination. It introduces ProFL, an algorithm that estimates the full performative gradient, combining robust local gradient estimation with a (possibly server-side) Jacobian estimate of how data shifts respond to model changes. The authors prove convergence under the Polyak-Lojasiewicz condition and demonstrate robustness to contamination, outperforming prior work that only reaches the performative stable point (PS) or relies on convex/noisy assumptions. The approach is validated on synthetic and real-features case studies, showing improved PO convergence, resilience to outliers, and reduced communication through adaptive sampling. Overall, ProFL advances practical, robust, distributed optimization under endogenous distribution shifts in FL settings.

Abstract

Performative prediction is a framework that captures distribution shifts that occur during the training of machine learning models due to their deployment. As the trained model is used, data generation causes the model to evolve, leading to deviations from the original data distribution. The impact of such model-induced distribution shifts in federated learning is increasingly likely to transpire in real-life use cases. A recently proposed approach extends performative prediction to federated learning with the resulting model converging to a performative stable point, which may be far from the performative optimal point. Earlier research in centralized settings has shown that the performative optimal point can be achieved under model-induced distribution shifts, but these approaches require the performative risk to be convex and the training data to be noiseless, assumptions often violated in realistic federated learning systems. This paper overcomes all of these shortcomings and proposes Performative Robust Optimal Federated Learning, an algorithm that finds performative optimal points in federated learning from noisy and contaminated data. We present the convergence analysis under the Polyak-Lojasiewicz condition, which applies to non-convex objectives. Extensive experiments on multiple datasets demonstrate the advantage of Robust Optimal Federated Learning over the state-of-the-art.

Figures (6)

• Figure 1: System model of performative prediction. The model $\theta$ influences the data distribution $D(\theta)$, forming a feedback loop that drives endogenous data shifts.
• Figure 2: (a) Robust gradient mitigates the impact of data contamination. (b) Estimation of $\frac{df_i}{d\theta}$ on the server outperforms local estimation in the large-client, limited-sample setting ($|\mathcal{V}| = 100$, $n_i = 60$). (c) Adaptive sampling (tolerances 0.05/0.1) reduces sample usage while maintaining accuracy.
• Figure 3: (a) Increasing sample size $n_i$ improves stability and convergence to the PO point. (b) A smaller learning rate $\eta$ slows convergence but yields solutions closer to the PO point. (c) A smaller estimation window $H$ produces more local estimates of $\frac{df_i}{d\theta}$, reducing final error at the cost of slower convergence.
• Figure 4: Convergence comparison between ProFL and PFL on both synthetic and real-world datasets. ProFL consistently achieves higher accuracy with fewer communication rounds.
• Figure 5: Convergence for pricing with dynamic contribution and regression with dynamic contribution. ProFL approaches the performative optimum more closely than PFL, with faster convergence and reduced oscillation, despite a gap due to non-linear $f_i(\cdot)$.

Theorems & Definitions (17)

• Remark 2.1
• Remark 4.5
• Lemma 4.6
• Lemma 4.7: Estimation error of $\frac{\widehat{df_i}}{d\theta}$
• Lemma 4.8: Estimation error of $\nabla \mathcal{L}_{i,1}$
• Lemma 4.9: Estimation error of $\nabla \mathcal{L}_{i,2}$
• Theorem 4.10: Convergence rate
• Corollary 4.11
• proof
• proof