Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PrivDFS: Private Inference via Distributed Feature Sharing against Data Reconstruction Attacks

Zihan Liu, Jiayi Wen, Junru Wu, Xuyang Zou, Shouhong Tan, Zhirun Zheng, Cheng Huang

TL;DR

PrivDFS addresses input privacy in cloud-based image classification by replacing a single exposed intermediate representation with distributed feature shares across an honest-majority server set. A learnable Distributed Feature Sharing module creates sparse, largely non-overlapping shares that are processed independently, with a lightweight fusion module on the client to recover accuracy. The approach substantially reduces diffusion-based data reconstruction attack fidelity while keeping accuracy close to non-private split inference and achieving strong efficiency due to minimal client computation. It demonstrates architectural generality by applying to CNNs and Vision Transformers and scales to ImageNet-1K with favorable privacy-utility-efficiency trade-offs.

Abstract

In this paper, we introduce PrivDFS, a distributed feature-sharing framework for input-private inference in image classification. A single holistic intermediate representation in split inference gives diffusion-based Data Reconstruction Attacks (DRAs) sufficient signal to reconstruct the input with high fidelity. PrivDFS restructures this vulnerability by fragmenting the representation and processing the fragments independently across a majority-honest set of servers. As a result, each branch observes only an incomplete and reconstruction-insufficient view of the input. To realize this, PrivDFS employs learnable binary masks that partition the intermediate representation into sparse and largely non-overlapping feature shares, each processed by a separate server, while a lightweight fusion module aggregates their predictions on the client. This design preserves full task accuracy when all branches are combined, yet sharply limits the reconstructive power available to any individual server. PrivDFS applies seamlessly to both ResNet-based CNNs and Vision Transformers. Across CIFAR-10/100, CelebA, and ImageNet-1K, PrivDFS induces a pronounced collapse in DRA performance, e.g., on CIFAR-10, PSNR drops from 23.25 -> 12.72 and SSIM from 0.963 -> 0.260, while maintaining accuracy within 1% of non-private split inference. These results establish structural feature partitioning as a practical and architecture-agnostic approach to reducing reconstructive leakage in cloud-based vision inference.

PrivDFS: Private Inference via Distributed Feature Sharing against Data Reconstruction Attacks

TL;DR

PrivDFS addresses input privacy in cloud-based image classification by replacing a single exposed intermediate representation with distributed feature shares across an honest-majority server set. A learnable Distributed Feature Sharing module creates sparse, largely non-overlapping shares that are processed independently, with a lightweight fusion module on the client to recover accuracy. The approach substantially reduces diffusion-based data reconstruction attack fidelity while keeping accuracy close to non-private split inference and achieving strong efficiency due to minimal client computation. It demonstrates architectural generality by applying to CNNs and Vision Transformers and scales to ImageNet-1K with favorable privacy-utility-efficiency trade-offs.

Abstract

In this paper, we introduce PrivDFS, a distributed feature-sharing framework for input-private inference in image classification. A single holistic intermediate representation in split inference gives diffusion-based Data Reconstruction Attacks (DRAs) sufficient signal to reconstruct the input with high fidelity. PrivDFS restructures this vulnerability by fragmenting the representation and processing the fragments independently across a majority-honest set of servers. As a result, each branch observes only an incomplete and reconstruction-insufficient view of the input. To realize this, PrivDFS employs learnable binary masks that partition the intermediate representation into sparse and largely non-overlapping feature shares, each processed by a separate server, while a lightweight fusion module aggregates their predictions on the client. This design preserves full task accuracy when all branches are combined, yet sharply limits the reconstructive power available to any individual server. PrivDFS applies seamlessly to both ResNet-based CNNs and Vision Transformers. Across CIFAR-10/100, CelebA, and ImageNet-1K, PrivDFS induces a pronounced collapse in DRA performance, e.g., on CIFAR-10, PSNR drops from 23.25 -> 12.72 and SSIM from 0.963 -> 0.260, while maintaining accuracy within 1% of non-private split inference. These results establish structural feature partitioning as a practical and architecture-agnostic approach to reducing reconstructive leakage in cloud-based vision inference.

Paper Structure

This paper contains 16 sections, 6 equations, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Threat Model
  3. Proposed Framework: PrivDFS
  4. Framework Overview
  5. Module Design: DFS and Fusion
  6. Adaptation to Vision Transformers
  7. Evaluation
  8. Experimental Settings
  9. Experiments on Privacy
  10. Scalability to ImageNet-1K.
  11. Experiments on Efficiency
  12. PrivDFS-ViT Performance
  13. Extended Experiments
  14. Scalability with Server Count.
  15. Inference Incapability of a Single Share.
  16. ...and 1 more sections

Figures (5)

  • Figure 1: Comparison of architectural paradigms: (a) Conventional single-server split inference. (b) PrivDFS.
  • Figure 2: Illustration of the PrivDFS framework for an example configuration with $K=3$ servers.
  • Figure 3: Qualitative reconstruction results under DRA on CIFAR-10 and CelebA. Columns correspond to different defense methods.
  • Figure 4: Qualitative reconstruction results on CIFAR-10 under DRA, comparing standard SI with PrivDFS-ViT.
  • Figure 5: Impact of the number of server models ($K$) on task accuracy: PrivDFS scales well with more branches.