Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Quantifying Privacy Leakage in Split Inference via Fisher-Approximated Shannon Information Analysis

Ruijun Deng, Zhihui Lu, Qiang Duan, Shijing Hu

TL;DR

The paper addresses privacy leakage in split inference by formalizing an information-theoretic framework that quantifies adversary certainty via negative conditional entropy and worst-case bounds. It introduces FSInfo, a Fisher information–based, Shannon-information–level metric that is computable in practice by Gaussian surrogates, providing a robust proxy for leakage from smashed data. Building on FSInfo, the authors design FSInfoGuard, a closed-form Gaussian noise defense calibrated to a target FSInfo to achieve favorable privacy–utility trade-offs, validated across multi-modal datasets and models. The results demonstrate FSInfo's effectiveness as both an assessment tool and a design signal for defense strategies, with practical guidance on data complexity, model size, and overfitting as leakage determinants.

Abstract

Split inference (SI) partitions deep neural networks into distributed sub-models, enabling collaborative learning without directly sharing raw data. However, SI remains vulnerable to Data Reconstruction Attacks (DRAs), where adversaries exploit exposed smashed data to recover private inputs. Despite substantial progress in attack-defense methodologies, the fundamental quantification of privacy risks is still underdeveloped. This paper establishes an information-theoretic framework for privacy leakage in SI, defining leakage as the adversary's certainty and deriving both average-case and worst-case error lower bounds. We further introduce Fisher-approximated Shannon information (FSInfo), a new privacy metric based on Fisher Information (FI) that enables operational and tractable computation of privacy leakage. Building on this metric, we develop FSInfoGuard, a defense mechanism that achieves a strong privacy-utility tradeoff. Our empirical study shows that FSInfo is an effective privacy metric across datasets, models, and defense strengths, providing accurate privacy estimates that support the design of defense methods outperforming existing approaches in both privacy protection and utility preservation. The code is available at https://github.com/SASA-cloud/FSInfo.

Quantifying Privacy Leakage in Split Inference via Fisher-Approximated Shannon Information Analysis

TL;DR

The paper addresses privacy leakage in split inference by formalizing an information-theoretic framework that quantifies adversary certainty via negative conditional entropy and worst-case bounds. It introduces FSInfo, a Fisher information–based, Shannon-information–level metric that is computable in practice by Gaussian surrogates, providing a robust proxy for leakage from smashed data. Building on FSInfo, the authors design FSInfoGuard, a closed-form Gaussian noise defense calibrated to a target FSInfo to achieve favorable privacy–utility trade-offs, validated across multi-modal datasets and models. The results demonstrate FSInfo's effectiveness as both an assessment tool and a design signal for defense strategies, with practical guidance on data complexity, model size, and overfitting as leakage determinants.

Abstract

Split inference (SI) partitions deep neural networks into distributed sub-models, enabling collaborative learning without directly sharing raw data. However, SI remains vulnerable to Data Reconstruction Attacks (DRAs), where adversaries exploit exposed smashed data to recover private inputs. Despite substantial progress in attack-defense methodologies, the fundamental quantification of privacy risks is still underdeveloped. This paper establishes an information-theoretic framework for privacy leakage in SI, defining leakage as the adversary's certainty and deriving both average-case and worst-case error lower bounds. We further introduce Fisher-approximated Shannon information (FSInfo), a new privacy metric based on Fisher Information (FI) that enables operational and tractable computation of privacy leakage. Building on this metric, we develop FSInfoGuard, a defense mechanism that achieves a strong privacy-utility tradeoff. Our empirical study shows that FSInfo is an effective privacy metric across datasets, models, and defense strengths, providing accurate privacy estimates that support the design of defense methods outperforming existing approaches in both privacy protection and utility preservation. The code is available at https://github.com/SASA-cloud/FSInfo.

Paper Structure

This paper contains 29 sections, 7 theorems, 22 equations, 12 figures, 3 tables.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORK
  3. PRELIMINARIES
  4. Split Inference Pipeline
  5. Threat Model
  6. Adversary's Goal.
  7. Adversary's Knowledge & Capability.
  8. Privacy Leakage Quantification
  9. Privacy and Privacy Leakage
  10. Privacy Leakage Formulation
  11. Computing Privacy Leakage
  12. Guiding Calculation of Defensive Noise
  13. Experiments
  14. Setups
  15. Datasets and Models
  16. ...and 14 more sections

Key Result

Theorem 4.2

( Average-Case Reconstruction Error Bound). For two random variables raw input $X$ and reconstructed input $\hat{X}$ with the same dimension $d_x$, we have: where the lower bound of the adversary's error is smaller under a higher privacy leakage. The proof is provided in the Appendix.

Figures (12)

  • Figure 1: The information flow in a split inference system and the data reconstruction attack.
  • Figure 2: The proposed FSInfo privacy metric.
  • Figure 3: MSE vs. FSInfo across different datasets and models.
  • Figure 4: FSInfo under different defenses and strengths.
  • Figure 5: Visualization of raw input (left) and the reconstructed input of ResNet-18 on CIFAR-10 among different split points vs. their FSInfo.
  • ...and 7 more figures

Theorems & Definitions (13)

  • Definition 4.1
  • Theorem 4.2
  • Definition 4.3
  • Theorem 4.4
  • Definition 4.5
  • Theorem 4.6
  • Theorem 4.7
  • Definition 4.8
  • Theorem 1
  • Lemma 2
  • ...and 3 more