FedAdOb: Privacy-Preserving Federated Deep Learning with Adaptive Obfuscation

TL;DR

FedAdOb introduces passport-based adaptive obfuscation for federated deep learning, aiming to protect private features and labels in both horizontal and vertical FL without sacrificing performance. By embedding learnable passports into bottom/top layers and generating them randomly, the method achieves strong privacy guarantees while co-adapting with model optimization, supported by theoretical hardness results (Theorems 1–2) and empirical CAP-guided evaluations. The approach significantly improves the privacy-utility trade-off relative to fixed obfuscation and several baseline defenses across diverse datasets and architectures, with only modest computational overhead. This work advances practical privacy-preserving FL by enabling adaptable, efficiently trainable obfuscation that defends against feature- and label-leakage attacks in both HFL and VFL settings.

Abstract

Federated learning (FL) has emerged as a collaborative approach that allows multiple clients to jointly learn a machine learning model without sharing their private data. The concern about privacy leakage, albeit demonstrated under specific conditions, has triggered numerous follow-up research in designing powerful attacking methods and effective defending mechanisms aiming to thwart these attacking methods. Nevertheless, privacy-preserving mechanisms employed in these defending methods invariably lead to compromised model performances due to a fixed obfuscation applied to private data or gradients. In this article, we, therefore, propose a novel adaptive obfuscation mechanism, coined FedAdOb, to protect private data without yielding original model performances. Technically, FedAdOb utilizes passport-based adaptive obfuscation to ensure data privacy in both horizontal and vertical federated learning settings. The privacy-preserving capabilities of FedAdOb, specifically with regard to private features and labels, are theoretically proven through Theorems 1 and 2. Furthermore, extensive experimental evaluations conducted on various datasets and network architectures demonstrate the effectiveness of FedAdOb by manifesting its superior trade-off between privacy preservation and model performance, surpassing existing methods.

Figures (6)

• Figure 1: Adaptive obfuscation ($g_W(\cdot)$). We implement $g_W(\cdot)$ by inserting a passport layer into a normal neural network layer.
• Figure 2: The left sub-figure illustrates the FedAdOb for HFL, including adaptive obfuscation $g_\theta$ and federated model $f_\omega$. The right sub-figure illustrates the FedAdOb for the VFL setting, in which multiple passive parties and one active party collaboratively train a VFL model, where passive parties only have the private features $x$, whereas the active party has private labels $y$. Both the active party and the passive party adopt adaptive obfuscation by inserting passports into their models to protect features and labels.
• Figure 3: HFL tradeoff. Comparison of different defense methods in terms of their trade-offs between main task accuracy and data recovery error against WGI attack zhu2019dlg (the first line) and WMI attack he2019model (the second line) on LeNet-MNIST, AlexNet-CIFAR10, and ResNet-CIFAR100, respectively. Trade-off curves near the top right corner are preferred to faraway ones.
• Figure 4: VFL tradeoff. Comparison of different defense methods in terms of their trade-offs between main task accuracy and data (feature or label) recovery error against three attacks on LeNet-MNIST, AlexNet-CIFAR10, ResNet-CIFAR10, respectively and LeNet-ModelNet. BMI (the first column) and WMI (the second column) are feature reconstruction attacks, whereas Passive Model Completion (the third column) is a label inference attack. A better trade-off curve should be more toward the top-right corner of each figure.
• Figure 5: Original images and images reconstructed by CAFE attack in VFL for different defense mechanisms on LeNet-MNIST, AlexNet-CIFAR10, ResNet-CIFAR10 and LeNet-ModelNet respectively. From top to bottom, a row represents original image ($r1$), no defense ($r2$), InstaHide ($r3$), DP with noise level $0.2$ ($r4$) and $2$ ($r5$), Sparsification with sparsification level $0.5$ ($r6$) and $0.05$ ($r7$), and FedAdOb ($r8$).

Theorems & Definitions (21)

• Proposition 1
• Definition 1
• Theorem 1
• Theorem 2
• Proposition 2
• Definition 2: Calibrated Averaged Performance (CAP)
• Definition 1
• Lemma 1
• proof
• Remark 1