Federated Gaussian Process: Convergence, Automatic Personalization and Multi-fidelity Modeling
Xubo Yue, Raed Al Kontar
TL;DR
FGPR introduces a federated Gaussian process regression framework that blends FedAvg model aggregation with local SGD updates to learn a shared GP prior across clients. The authors prove convergence to a stationary point of the full data log-likelihood under kernels with exponential or polynomial eigendecay, with explicit error terms that reflect per-client batch sizes and communication scheme. A central finding is automatic personalization: predictions on each client are conditioned on both the learned global GP prior and the client’s own data, enabling strong performance even with heterogeneous data. Empirically, FGPR demonstrates privacy-preserving advantages in multi-fidelity modeling and achieves superior or competitive results across simulated and real-world tasks involving multi-fidelity data, NASA turbine degradation, and robotic inverse dynamics. The work provides a theoretical foundation for federated inference with correlated Gaussian processes and opens avenues for privacy-aware Bayesian meta-learning and transfer across devices.
Abstract
In this paper, we propose \texttt{FGPR}: a Federated Gaussian process ($\mathcal{GP}$) regression framework that uses an averaging strategy for model aggregation and stochastic gradient descent for local client computations. Notably, the resulting global model excels in personalization as \texttt{FGPR} jointly learns a global $\mathcal{GP}$ prior across all clients. The predictive posterior then is obtained by exploiting this prior and conditioning on local data which encodes personalized features from a specific client. Theoretically, we show that \texttt{FGPR} converges to a critical point of the full log-likelihood function, subject to statistical error. Through extensive case studies we show that \texttt{FGPR} excels in a wide range of applications and is a promising approach for privacy-preserving multi-fidelity data modeling.