Peak-Controlled Logits Poisoning Attack in Federated Distillation

TL;DR

This paper addresses the security vulnerability of Federated Distillation (FD) to logits-based poisoning attacks. It introduces Peak-Controlled Federated Distillation Logits Attack (PCFDLA), an enhancement of the prior FDLA method that strategically controls peak logits to mislead learning while preserving the attacker's ability to predict correct answers. A new victim-oriented metric is proposed to quantify the impact on attacked clients, and extensive experiments across CINIC-10, CIFAR-10, and SVHN show that PCFDLA significantly degrades victim model accuracy compared with baselines, with the strength of the attack controllable via a hyperparameter. The results highlight a concrete security risk in FD pipelines and emphasize the need for defenses against logits-level manipulation in distributed distillation systems.

Abstract

Federated Distillation (FD) offers an innovative approach to distributed machine learning, leveraging knowledge distillation for efficient and flexible cross-device knowledge transfer without necessitating the upload of extensive model parameters to a central server. While FD has gained popularity, its vulnerability to poisoning attacks remains underexplored. To address this gap, we previously introduced FDLA (Federated Distillation Logits Attack), a method that manipulates logits communication to mislead and degrade the performance of client models. However, the impact of FDLA on participants with different identities and the effects of malicious modifications at various stages of knowledge transfer remain unexplored. To this end, we present PCFDLA (Peak-Controlled Federated Distillation Logits Attack), an advanced and more stealthy logits poisoning attack method for FD. PCFDLA enhances the effectiveness of FDLA by carefully controlling the peak values of logits to create highly misleading yet inconspicuous modifications. Furthermore, we introduce a novel metric for better evaluating attack efficacy, demonstrating that PCFDLA maintains stealth while being significantly more disruptive to victim models compared to its predecessors. Experimental results across various datasets confirm the superior impact of PCFDLA on model accuracy, solidifying its potential threat in federated distillation systems.

Figures (6)

• Figure 1: Illustration of how FDLA attackers manipulates uploaded knowledge.
• Figure 2: Illustration of how PCFDLA attackers manipulates uploaded knowledge.
• Figure 3: Convergence impact of three attack types on FD and FedCache in CIFAR-10 with 30% attackers, tracking average accuracy per communication round.
• Figure 4: Illustration of the distribution of malicious and non-malicious user models after PCA dimensionality reduction. The malicious users are marked in blue, while the non-malicious users are marked in orange.
• Figure 5: Illustration of the misleading effect of PCFDLA to client models. The vertical axis of the chart represents the probability statistics of the test results for cat samples, while the horizontal axis represents the categories of the test results.