Joint Graph Estimation and Signal Restoration for Robust Federated Learning

TL;DR

Experimental results on MNIST and CIFAR10 datasets show that the proposed method outperforms existing approaches by up to 2-5% in classification accuracy under biased data distributions and noisy conditions.

Abstract

We propose a robust aggregation method for model parameters in federated learning (FL) under noisy communications. FL is a distributed machine learning paradigm in which a central server aggregates local model parameters from multiple clients. These parameters are often noisy and/or have missing values during data collection, training, and communication between the clients and server. This may cause a considerable drop in model accuracy. To address this issue, we learn a graph that represents pairwise relationships between model parameters of the clients during aggregation. We realize it with a joint problem of graph learning and signal (i.e., model parameters) restoration. The problem is formulated as a difference-of-convex (DC) optimization, which is efficiently solved via a proximal DC algorithm. Experimental results on MNIST and CIFAR-10 datasets show that the proposed method outperforms existing approaches by up to $2$--$5\%$ in classification accuracy under biased data distributions and noisy conditions.

Figures (4)

• Figure 1: Overviews of the standard and graph-based FL.
• Figure 2: Overview of the proposed FL.
• Figure 3: Average accuracies at each communication round on CIFAR-10 under a noise level $s=0.2$.
• Figure 4: Average accuracy of the proposed method for missing rates from 0 to 0.1 in 0.01 increments ($s=0.1$).