Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An Efficient Difference-of-Convex Solver for Privacy Funnel

Teng-Hui Huang, Hesham El Gamal

TL;DR

This work tackles the Privacy Funnel problem by exploiting a difference-of-convex (DC) structure to derive efficient solvers. The authors propose a DC separation with closed-form updates, yielding a practical algorithm that converges to local stationary points for known distributions and extends to unknown distributions via a variational-DC hybrid that preserves PF’s Markov constraint during inference. Empirical results on MNIST and Fashion-MNIST show improved privacy-utility trade-offs, with reduced adversary leakage at comparable reconstruction quality and increased training efficiency compared to baselines. The approach provides a scalable framework for PF in both fully specified and data-driven settings, enabling robust private-data releases in practical deployments.

Abstract

We propose an efficient solver for the privacy funnel (PF) method, leveraging its difference-of-convex (DC) structure. The proposed DC separation results in a closed-form update equation, which allows straightforward application to both known and unknown distribution settings. For known distribution case, we prove the convergence (local stationary points) of the proposed non-greedy solver, and empirically show that it outperforms the state-of-the-art approaches in characterizing the privacy-utility trade-off. The insights of our DC approach apply to unknown distribution settings where labeled empirical samples are available instead. Leveraging the insights, our alternating minimization solver satisfies the fundamental Markov relation of PF in contrast to previous variational inference-based solvers. Empirically, we evaluate the proposed solver with MNIST and Fashion-MNIST datasets. Our results show that under a comparable reconstruction quality, an adversary suffers from higher prediction error from clustering our compressed codes than that with the compared methods. Most importantly, our solver is independent to private information in inference phase contrary to the baselines.

An Efficient Difference-of-Convex Solver for Privacy Funnel

TL;DR

This work tackles the Privacy Funnel problem by exploiting a difference-of-convex (DC) structure to derive efficient solvers. The authors propose a DC separation with closed-form updates, yielding a practical algorithm that converges to local stationary points for known distributions and extends to unknown distributions via a variational-DC hybrid that preserves PF’s Markov constraint during inference. Empirical results on MNIST and Fashion-MNIST show improved privacy-utility trade-offs, with reduced adversary leakage at comparable reconstruction quality and increased training efficiency compared to baselines. The approach provides a scalable framework for PF in both fully specified and data-driven settings, enabling robust private-data releases in practical deployments.

Abstract

We propose an efficient solver for the privacy funnel (PF) method, leveraging its difference-of-convex (DC) structure. The proposed DC separation results in a closed-form update equation, which allows straightforward application to both known and unknown distribution settings. For known distribution case, we prove the convergence (local stationary points) of the proposed non-greedy solver, and empirically show that it outperforms the state-of-the-art approaches in characterizing the privacy-utility trade-off. The insights of our DC approach apply to unknown distribution settings where labeled empirical samples are available instead. Leveraging the insights, our alternating minimization solver satisfies the fundamental Markov relation of PF in contrast to previous variational inference-based solvers. Empirically, we evaluate the proposed solver with MNIST and Fashion-MNIST datasets. Our results show that under a comparable reconstruction quality, an adversary suffers from higher prediction error from clustering our compressed codes than that with the compared methods. Most importantly, our solver is independent to private information in inference phase contrary to the baselines.
Paper Structure (17 sections, 2 theorems, 3 equations, 5 figures, 1 table)

This paper contains 17 sections, 2 theorems, 3 equations, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. A Difference-of-Convex Algorithm for Privacy Funnel
  3. Extension to Unknown Distributions
  4. Evaluation
  5. Known Distribution
  6. Unknown Distribution
  7. Datasets
  8. Network Architecture
  9. Results
  10. Conclusions
  11. Derivation for a known joint distribution
  12. Alternative Approach for Sparse Recovery
  13. Derivation for unknown joint distribution
  14. Proof of Theorem \ref{['thm:conv_discrete_kn']}
  15. Experiment Details
  16. ...and 2 more sections

Key Result

Theorem 1

For both implementations eq:lasso_pf ($q=2$) and eq:lse_q1, the sequence $\{\boldsymbol{A}_x\boldsymbol{p}_{z|x}^k\}_{k\in\mathbb{N}}$, obtained from eq:DCA_for_pf, converges to a stationary point $\boldsymbol{A}_x\boldsymbol{p}_{z|x}^*$ such that $\nabla f(\boldsymbol{p}^*_{z|x})=\nabla g(\boldsymb

Figures (5)

  • Figure 1: Comparing the characterization of the privacy-utility trade-off between the proposed DCA solvers and the state-of-the-art method submodpf2019. The joint distribution $P(X,Y)$ is known and is given by \ref{['eq:syn_discrete']}.
  • Figure 2: Privacy-utility trade-off in unknown joint distribution settings. The proposed DCA solver is compared to the CPF rodriguez2021variational and DVPF razeghi2024deep baselines.
  • Figure 3: Privacy-utility trade-off of the MNIST dataset in different code length. Here, $\boldsymbol{x}\in\mathbb{R}^{784}$ whereas $\boldsymbol{z}\in\mathbb{R}^d$.
  • Figure 4: Testing data versus reconstruction from the DCA solver. PSNR=$24$ dB, Adversary success rate=$0.57$
  • Figure 5: Visualization of the codes from DCA solver. PSNR=$24$dB, Adversary success rate=$0.57$ and code length $d_z=256$. Centers of each class are labeled.

Theorems & Definitions (2)

  • Theorem 1
  • Lemma 1: Restricted Convexity