Maximizing Uncertainty for Federated learning via Bayesian Optimisation-based Model Poisoning

TL;DR

This work tackles the trustworthiness of federated learning by focusing on uncertainty and introduces Delphi, a model-poisoning attack that maximises global output uncertainty through targeted perturbations of the first hidden layer. It develops two optimization frameworks, Delphi-BO (Bayesian Optimisation) and Delphi-LSTR (Least Squares Trust Region), with a KL-divergence objective to drive the poisoned weights while constraining perturbations. The authors provide a mathematical bound on attack effectiveness and demonstrate through CIFAR-10/100 experiments that Delphi-BO more effectively increases uncertainty than Delphi-LSTR, with dynamic neuron selection further enhancing impact; they also assess robustness under Krum aggregation. The findings reveal a notable vulnerability in FL to uncertainty-driven poisoning, informing defense design and motivating future work on broader aggregations and DL models.

Abstract

As we transition from Narrow Artificial Intelligence towards Artificial Super Intelligence, users are increasingly concerned about their privacy and the trustworthiness of machine learning (ML) technology. A common denominator for the metrics of trustworthiness is the quantification of uncertainty inherent in DL algorithms, and specifically in the model parameters, input data, and model predictions. One of the common approaches to address privacy-related issues in DL is to adopt distributed learning such as federated learning (FL), where private raw data is not shared among users. Despite the privacy-preserving mechanisms in FL, it still faces challenges in trustworthiness. Specifically, the malicious users, during training, can systematically create malicious model parameters to compromise the models predictive and generative capabilities, resulting in high uncertainty about their reliability. To demonstrate malicious behaviour, we propose a novel model poisoning attack method named Delphi which aims to maximise the uncertainty of the global model output. We achieve this by taking advantage of the relationship between the uncertainty and the model parameters of the first hidden layer of the local model. Delphi employs two types of optimisation , Bayesian Optimisation and Least Squares Trust Region, to search for the optimal poisoned model parameters, named as Delphi-BO and Delphi-LSTR. We quantify the uncertainty using the KL Divergence to minimise the distance of the predictive probability distribution towards an uncertain distribution of model output. Furthermore, we establish a mathematical proof for the attack effectiveness demonstrated in FL. Numerical results demonstrate that Delphi-BO induces a higher amount of uncertainty than Delphi-LSTR highlighting vulnerability of FL systems to model poisoning attacks.

Figures (9)

• Figure 1: A Federated Learning system, where there is $N$ number of Benign clients and $A$ number of Malicious clients. The Clients are using their own dataset to train their own model. The malicious clients are either train their own model or poison the neurons of the first hidden layer using Delphi. Delphi-BO collects data points to create a surrogate model (i) & (ii), and finds a new sampling point using an acquisition function (iii). Then modifies with the new parameters $\theta_{t+1}$ the model, and observes the uncertainty (iv). This is repeated for $T$ amount of runs. Delphi-LSTR, search for a new $\theta_{t+1}$ until convergence (i), via solving a subproblem that is to find a small step $s_t$ (ii).Then modifies with the new parameters $\theta_{t+1}$ the model, and observes the uncertainty (iii)
• Figure 2: Comparison between Bayesian Optimisation and Least Squares with fixed neuron selection scheme and datasets of CIFAR10 and CIFAR100. We show the effects of mean predictive confidence, accuracy and entropy when attacking 5 neurons.
• Figure 3: Comparison between Bayesian Optimisation and Least Squares, with varying number of malicious users and neurons with fixed neuron selection scheme. We show the effects of mean predictive confidence.
• Figure 4: Comparison of manipulating with fixed and dynamic set of neurons, in the setting of IID data distribution
• Figure 5: Comparison of manipulating a fixed or dynamic set of neurons, in the setting of Imbalanced data distribution