Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Residue-based Label Protection Mechanisms in Vertical Logistic Regression

Juntao Tan, Lan Zhang, Yang Liu, Anran Li, Ye Wu

TL;DR

The paper addresses a residue-based label inference vulnerability in vertical federated learning for logistic regression and presents three protection strategies. It introduces $M_{add}$ (additive noise), $M_{mult}$ (multiplicative noise), and $M_{hybrid}$ (local differential privacy with homomorphic encryption) to safeguard residues, providing $\\epsilon$-LDP guarantees and cryptographic protection. Empirical results across four datasets show that $M_{add}$ and $M_{mult}$ incur minimal accuracy loss at moderate privacy budgets, while $M_{hybrid}$ achieves lossless protection with at most a modest computational overhead (up to $1.8\times$). The work offers practical privacy-utility trade-offs for robust vertical LR deployments and demonstrates the feasibility of protecting private labels without sacrificing model performance in real-world settings.

Abstract

Federated learning (FL) enables distributed participants to collaboratively learn a global model without revealing their private data to each other. Recently, vertical FL, where the participants hold the same set of samples but with different features, has received increased attention. This paper first presents one label inference attack method to investigate the potential privacy leakages of the vertical logistic regression model. Specifically, we discover that the attacker can utilize the residue variables, which are calculated by solving the system of linear equations constructed by local dataset and the received decrypted gradients, to infer the privately owned labels. To deal with this, we then propose three protection mechanisms, e.g., additive noise mechanism, multiplicative noise mechanism, and hybrid mechanism which leverages local differential privacy and homomorphic encryption techniques, to prevent the attack and improve the robustness of the vertical logistic regression. model. Experimental results show that both the additive noise mechanism and the multiplicative noise mechanism can achieve efficient label protection with only a slight drop in model testing accuracy, furthermore, the hybrid mechanism can achieve label protection without any testing accuracy degradation, which demonstrates the effectiveness and efficiency of our protection techniques

Residue-based Label Protection Mechanisms in Vertical Logistic Regression

TL;DR

The paper addresses a residue-based label inference vulnerability in vertical federated learning for logistic regression and presents three protection strategies. It introduces (additive noise), (multiplicative noise), and (local differential privacy with homomorphic encryption) to safeguard residues, providing -LDP guarantees and cryptographic protection. Empirical results across four datasets show that and incur minimal accuracy loss at moderate privacy budgets, while achieves lossless protection with at most a modest computational overhead (up to ). The work offers practical privacy-utility trade-offs for robust vertical LR deployments and demonstrates the feasibility of protecting private labels without sacrificing model performance in real-world settings.

Abstract

Federated learning (FL) enables distributed participants to collaboratively learn a global model without revealing their private data to each other. Recently, vertical FL, where the participants hold the same set of samples but with different features, has received increased attention. This paper first presents one label inference attack method to investigate the potential privacy leakages of the vertical logistic regression model. Specifically, we discover that the attacker can utilize the residue variables, which are calculated by solving the system of linear equations constructed by local dataset and the received decrypted gradients, to infer the privately owned labels. To deal with this, we then propose three protection mechanisms, e.g., additive noise mechanism, multiplicative noise mechanism, and hybrid mechanism which leverages local differential privacy and homomorphic encryption techniques, to prevent the attack and improve the robustness of the vertical logistic regression. model. Experimental results show that both the additive noise mechanism and the multiplicative noise mechanism can achieve efficient label protection with only a slight drop in model testing accuracy, furthermore, the hybrid mechanism can achieve label protection without any testing accuracy degradation, which demonstrates the effectiveness and efficiency of our protection techniques
Paper Structure (27 sections, 16 equations, 3 figures, 4 tables)

This paper contains 27 sections, 16 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Background and Preliminaries
  3. Logistic Regression
  4. Homomorphic Encryption
  5. Local Differential Privacy
  6. Problem Statement
  7. System Model
  8. Adversary Model
  9. Problem Formulation
  10. Residue Protecting Mechanisms
  11. Additive Noise Mechanism $\mathcal{M}_{add}$
  12. Multiplicative Noise Mechanism $\mathcal{M}_{mult}$
  13. Hybrid Mechanism $\mathcal{M}_{hybrid}$ based on LDP and HE
  14. Training Protocol of $\mathcal{M}_{hybrid}$
  15. Security Analysis
  16. ...and 12 more sections

Figures (3)

  • Figure 1: Global model performance w.r.t.$b_1$, where we fix $b_2=10$ and $\epsilon=10$.
  • Figure 2: Global model performance w.r.t.$b_2$, where we fix $b_1=0.1$ and $\epsilon=10$.
  • Figure 3: Global model performance w.r.t.$\epsilon$, where we fix $b_1=0.1$ and $b_2=10$.