Federated Automatic Latent Variable Selection in Multi-output Gaussian Processes

TL;DR

This work tackles learning with multi-output Gaussian processes in a federated setting by automatically selecting the number of shared latent functions. It introduces a hierarchical LMC augmented with spike-and-slab priors on latent-coefficients and develops a variational inference framework compatible with federated optimization, enabling units to jointly infer latent patterns without sharing raw data. The method uses inducing points to reduce complexity and an efficient scheme for incorporating new units, achieving strong predictive performance while preserving privacy. Through simulations and real-world case studies on climate and battery data, the approach demonstrates accurate latent-function selection, competitive accuracy with centralized baselines, and practical feasibility for scalable, privacy-preserving applications.

Abstract

This paper explores a federated learning approach that automatically selects the number of latent processes in multi-output Gaussian processes (MGPs). The MGP has seen great success as a transfer learning tool when data is generated from multiple sources/units/entities. A common approach in MGPs to transfer knowledge across units involves gathering all data from each unit to a central server and extracting common independent latent processes to express each unit as a linear combination of the shared latent patterns. However, this approach poses key challenges in (i) determining the adequate number of latent processes and (ii) relying on centralized learning which leads to potential privacy risks and significant computational burdens on the central server. To address these issues, we propose a hierarchical model that places spike-and-slab priors on the coefficients of each latent process. These priors help automatically select only needed latent processes by shrinking the coefficients of unnecessary ones to zero. To estimate the model while avoiding the drawbacks of centralized learning, we propose a variational inference-based approach, that formulates model inference as an optimization problem compatible with federated settings. We then design a federated learning algorithm that allows units to jointly select and infer the common latent processes without sharing their data. We also discuss an efficient learning approach for a new unit within our proposed federated framework. Simulation and case studies on Li-ion battery degradation and air temperature data demonstrate the advantageous features of our proposed approach.

Figures (6)

• Figure 1: Our hierarchical modeling for LMC with spike-and-slab priors and its structural correspondence to connected systems.
• Figure 2: Predictions of the proposed and benchmark models.
• Figure 3: Predictions for the new units using the proposed new unit learning strategy.
• Figure 4: Predictive results for the air temperature data. The first row illustrates predictive curves for Sotonmet, Bramblemet, and Chimet; and the second row presents predictions for Cambermet based on the new unit learning approach.
• Figure 5: Estimated latent functions and their selection for the air temperature data. The latent function coefficients are regarded as zero if their absolute values are less than $1\times10^{-10}$.