Enhancing Robustness of Federated Learning via Server Learning

Abstract

This paper explores the use of server learning for enhancing the robustness of federated learning against malicious attacks even when clients' training data are not independent and identically distributed. We propose a heuristic algorithm that uses server learning and client update filtering in combination with geometric median aggregation. We demonstrate via experiments that this approach can achieve significant improvement in model accuracy even when the fraction of malicious clients is high, even more than $50\%$ in some cases, and the dataset utilized by the server is small and could be synthetic with its distribution not necessarily close to that of the clients' aggregated data.

Figures (9)

• Figure 1: Left: EMNIST training examples. Right: Server's synthetic examples.
• Figure 2: Left: CIFAR-10 training examples. Right: Server's STL-10 examples.
• Figure 3: Average accuracy during training on EMNIST-Dir$(0.3)$ using either Avg or GeoMed as the model aggregation step and without filtering. The case without SL is labeled as $\texttt{Avg}, \gamma =0$. We vary the value of $\eta_g \in\{1,2\}$ when using SL with $\gamma=0.1$. Shaded areas represent min-max values over 3 runs.
• Figure 4: Accuracy vs. malicious fraction $\beta$ when using Geometric Median for model aggregation but without SL.
• Figure 5: Test accuracy vs. training round on CIFAR-10 Dir$(0.3)$ using GeoMed aggregation, without SL, and with 0F (blue), AF (orange), LF (green).