Plug In and Learn: Federated Intelligence over a Smart Grid of Models

TL;DR

This work tackles federated learning over networks of heterogeneous data and models by introducing a graph-regularized, model-agnostic framework. The core idea is to couple local learners through a generalized total variation (GTV) penalty on an empirical graph $\mathcal{G}$, using a shared public unlabeled dataset $\mathcal{P}$ to evaluate discrepancies between neighboring models. The proposed method, FedRelax, supports both non-parametric and parametric local models and optimizes a regularized empirical risk via block-coordinate minimization, enabling decentralized updates without centralized parameter sharing. Empirical results on synthetic clustered and non-i.i.d. data demonstrate improved local performance and resilience to high $d/m$ regimes, with denoising and clustering effects observable across linear and tree-based models. Overall, FedRelax provides a flexible, privacy-preserving pathway to cooperative learning across diverse local learners by leveraging network structure and public data, with theoretical and empirical insights into how graph connectivity and regularization influence model variation.

Abstract

We present a model-agnostic federated learning method that mirrors the operation of a smart power grid: diverse local models, like energy prosumers, train independently on their own data while exchanging lightweight signals to coordinate with statistically similar peers. This coordination is governed by a graph-based regularizer that encourages connected models to produce similar predictions on a shared, public unlabeled dataset. The resulting method is a flexible instance of regularized empirical risk minimization and supports a wide variety of local models - both parametric and non-parametric - provided they can be trained via regularized loss minimization. Such training is readily supported by standard ML libraries including scikit-learn, Keras, and PyTorch.

Figures (5)

• Figure 1: FedRelax ${\rm MSE}^{(\widehat{{\bf w}})}$ on a clustered dataset with linear regressors as models. The code to generate the figure is available at https://github.com/shamPJ/het-FL/blob/main/src/FedRelax.ipynb.
• Figure 2: ${\rm MSE}^{(val)}$, normalized by Oracle, incurred by FedRelax on a clustered dataset and decision tree regressors as models. The y-axis in subplot (a) is on a logarithmic scale. The code to generate the figure is available at https://github.com/shamPJ/het-FL/blob/main/src/FedRelax.ipynb.
• Figure 3: FedRelax on non-i.i.d. datasets and linear regressors as models, ${\rm MSE}^{(val)}$ is normalized by the ${\rm MSE}^{(val)}$ incurred by the model trained locally ($\alpha=0$). In the first two subplots, the noise level is fixed at $\sigma=0$, while in the last subplot, $d/m_{i}=1$. The code to generate the figure is available at https://github.com/shamPJ/het-FL/blob/main/src/FedRelax-noniid.ipynb.
• Figure 4: FedRelax on non-i.i.d. datasets with decision tree regressors as models, normalized ${\rm MSE}^{(val)}$. The y-axis is on a logarithmic scale. In the first two subplots, the noise level is fixed at $\sigma=0$, while in the last subplot, $d/m_{i}=1$. The code to generate the figure is available at https://github.com/shamPJ/het-FL/blob/main/src/FedRelax-noniid.ipynb.
• Figure 5: Variation $\sum_{i \in \mathcal{C}} \left\Vert {\widetilde{{\bf w}}^{(i)} } \right\Vert_{2}^{2}$ and its bound under FedRelax on single-cluster dataset. Results are averaged over 50 runs. The y-axis is on a logarithmic scale. The code to generate the figure is available at https://github.com/shamPJ/het-FL/blob/main/src/FedRelax_variation.ipynb.