Training on Fake Labels: Mitigating Label Leakage in Split Learning via Secure Dimension Transformation

TL;DR

This paper proposes a novel two-party split learning method to defend against existing label inference attacks while maintaining the high utility of the learned models, and proposes a softmax-normalized Gaussian noise to mitigate privacy leakage and make the K unknowable to adversaries.

Abstract

Two-party split learning has emerged as a popular paradigm for vertical federated learning. To preserve the privacy of the label owner, split learning utilizes a split model, which only requires the exchange of intermediate representations (IRs) based on the inputs and gradients for each IR between two parties during the learning process. However, split learning has recently been proven to survive label inference attacks. Though several defense methods could be adopted, they either have limited defensive performance or significantly negatively impact the original mission. In this paper, we propose a novel two-party split learning method to defend against existing label inference attacks while maintaining the high utility of the learned models. Specifically, we first craft a dimension transformation module, SecDT, which could achieve bidirectional mapping between original labels and increased K-class labels to mitigate label leakage from the directional perspective. Then, a gradient normalization algorithm is designed to remove the magnitude divergence of gradients from different classes. We propose a softmax-normalized Gaussian noise to mitigate privacy leakage and make our K unknowable to adversaries. We conducted experiments on real-world datasets, including two binary-classification datasets (Avazu and Criteo) and three multi-classification datasets (MNIST, FashionMNIST, CIFAR-10); we also considered current attack schemes, including direction, norm, spectral, and model completion attacks. The detailed experiments demonstrate our proposed method's effectiveness and superiority over existing approaches. For instance, on the Avazu dataset, the attack AUC of evaluated four prominent attacks could be reduced by 0.4532+-0.0127.

Figures (8)

• Figure 1: In the threat model, we assume the host to be the attacker who wants to get the private data label owned by the guest. This inference attack can be done through the inference attack from the backward gradient.
• Figure 2: This figure outlines the SecDT algorithm's training process, which includes dimension transformation to expand the label space, gradient normalization to standardize gradient magnitudes, and noise-based randomization to introduce uncertainty into the label dimensions.
• Figure 3: This figure presents the results of SecDT's effectiveness against label inference attacks compared to other defense schemes such as Marvell, DP, and MixPro. It shows SecDT's superior performance in reducing Attack AUC without compromising Test AUC.
• Figure 4: This figure evaluates SecDT's performance against model completion attacks, where an adversary obtains a bottom model and fine-tunes it with auxiliary labeled data. SecDT demonstrates robustness against such attacks, with less impact on performance than other defense schemes.
• Figure 5: Normalization plays a vital role in defending against norm and model completion attacks.