Defending the Edge: Representative-Attention Defense against Backdoor Attacks in Federated Learning

TL;DR

Backdoor attacks in federated learning enable persistent, trigger-based misclassifications without sacrificing clean accuracy. FeRA introduces an attention-inspired defense that shifts from anomaly-based to consistency-based detection, leveraging representation-space variance suppression and coordination signals across six metrics. The framework employs two complementary detectors—consistency analysis and norm-inflation detection—and an adaptive aggregation scheme to bound malicious influence. Empirical results across multiple datasets, architectures, and attack types show FeRA delivering about $1.67\%$ average Backdoor Accuracy with high clean accuracy, and robust performance under non-IID settings and adaptive threat models, highlighting its practical potential for secure FL deployments.

Abstract

Federated learning (FL) remains highly vulnerable to adaptive backdoor attacks that preserve stealth by closely imitating benign update statistics. Existing defenses predominantly rely on anomaly detection in parameter or gradient space, overlooking behavioral constraints that backdoor attacks must satisfy to ensure reliable trigger activation. These anomaly-centric methods fail against adaptive attacks that normalize update magnitudes and mimic benign statistical patterns while preserving backdoor functionality, creating a fundamental detection gap. To address this limitation, this paper introduces FeRA (Federated Representative Attention) -- a novel attention-driven defense that shifts the detection paradigm from anomaly-centric to consistency-centric analysis. FeRA exploits the intrinsic need for backdoor persistence across training rounds, identifying malicious clients through suppressed representation-space variance, an orthogonal property to traditional magnitude-based statistics. The framework conducts multi-dimensional behavioral analysis combining spectral and spatial attention, directional alignment, mutual similarity, and norm inflation across two complementary detection mechanisms: consistency analysis and norm-inflation detection. Through this mechanism, FeRA isolates malicious clients that exhibit low-variance consistency or magnitude amplification. Extensive evaluation across six datasets, nine attacks, and three model architectures under both Independent and Identically Distributed (IID) and non-IID settings confirm FeRA achieves superior backdoor mitigation. Under different non-IID settings, FeRA achieved the lowest average Backdoor Accuracy (BA), about 1.67% while maintaining high clean accuracy compared to other state-of-the-art defenses. The code is available at https://github.com/Peatech/FeRA_defense.git.

Figures (9)

• Figure 1: Centered kernel alignment (CKA) similarity matrix. Malicious clients increase their similarity to benign clients, reducing the separability required by metric-based detection approaches.
• Figure 2: Two-dimensional principal component analysis (2D PCA) of representation deltas. Malicious updates overlap with benign representation shifts, reducing geometric separation and weakening clustering-based detection.
• Figure 3: Representation variance distribution across benign and backdoored clients across $n=100$ rounds.
• Figure 4: Inter-client parameter similarity reveals coordination patterns. Heatmap shows pairwise cosine similarity for 10 clients (3 malicious, 7 benign) over 50 rounds. Malicious-malicious pairs (marked M) exhibit higher mean similarity compared to benign-benign pairs, indicating coordinated optimization toward shared backdoor objectives despite data heterogeneity.
• Figure 5: Spectral variance suppression across top-$k$ eigenvalues. Malicious clients (red) exhibit consistently lower eigenvalues $\lambda_1$ through $\lambda_5$ compared to benign clients (blue) across several rounds under attack on CIFAR-10.