Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Activity Recognition on Avatar-Anonymized Datasets with Masked Differential Privacy

David Schneider, Sina Sajadmanesh, Vikash Sehwag, Saquib Sarfraz, Rainer Stiefelhagen, Lingjuan Lyu, Vivek Sharma

TL;DR

An anonymization pipeline that replaces sensitive human subjects in video datasets with synthetic avatars within context is presented, employing a combined rendering and stable diffusion-based strategy and masked differential privacy ({MaskDP}) to protect non-anonymized but privacy sensitive background information.

Abstract

Privacy-preserving computer vision is an important emerging problem in machine learning and artificial intelligence. Prevalent methods tackling this problem use differential privacy (DP) or obfuscation techniques to protect the privacy of individuals. In both cases, the utility of the trained model is sacrificed heavily in this process. In this work, we present an anonymization pipeline that replaces sensitive human subjects in video datasets with synthetic avatars within context, employing a combined rendering and stable diffusion-based strategy. Additionally we propose masked differential privacy ({MaskDP}) to protect non-anonymized but privacy sensitive background information. MaskDP allows for controlling sensitive regions where differential privacy is applied, in contrast to applying DP on the entire input. This combined methodology provides strong privacy protection while minimizing the usual performance penalty of privacy preserving methods. Experiments on multiple challenging action recognition datasets demonstrate that our proposed techniques result in better utility-privacy trade-offs compared to standard differentially private training in the especially demanding $ε<1$ regime.

Activity Recognition on Avatar-Anonymized Datasets with Masked Differential Privacy

TL;DR

An anonymization pipeline that replaces sensitive human subjects in video datasets with synthetic avatars within context is presented, employing a combined rendering and stable diffusion-based strategy and masked differential privacy ({MaskDP}) to protect non-anonymized but privacy sensitive background information.

Abstract

Privacy-preserving computer vision is an important emerging problem in machine learning and artificial intelligence. Prevalent methods tackling this problem use differential privacy (DP) or obfuscation techniques to protect the privacy of individuals. In both cases, the utility of the trained model is sacrificed heavily in this process. In this work, we present an anonymization pipeline that replaces sensitive human subjects in video datasets with synthetic avatars within context, employing a combined rendering and stable diffusion-based strategy. Additionally we propose masked differential privacy ({MaskDP}) to protect non-anonymized but privacy sensitive background information. MaskDP allows for controlling sensitive regions where differential privacy is applied, in contrast to applying DP on the entire input. This combined methodology provides strong privacy protection while minimizing the usual performance penalty of privacy preserving methods. Experiments on multiple challenging action recognition datasets demonstrate that our proposed techniques result in better utility-privacy trade-offs compared to standard differentially private training in the especially demanding regime.

Paper Structure

This paper contains 25 sections, 5 theorems, 12 equations, 5 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Trasformer-based Action recognition.
  4. Privacy preserving action recognition.
  5. Differential Privacy in Machine Learning.
  6. Anonymization
  7. Masked Differentially Private Training
  8. Remark.
  9. Learning with MaskDP
  10. Experiments
  11. Datasets
  12. Implementation Details
  13. The Importance of Context
  14. MaskDP on Anonymized Activity Datasets
  15. Differential Privacy on Imbalanced Datasets
  16. ...and 10 more sections

Key Result

Theorem 1

Given a dataset $\mathcal{D}$ of size $N$, batch-size $B<N$, number of training epochs $T$, gradient clipping threshold $C>0$, and Gaussian noise standard deviation $\sigma>0$, alg:ModelTraining satisfies $(\epsilon, \delta)$-MaskDP for any given $\delta\in(0,1)$, where: providing that the labels are not protected, i.e., assuming the label to be the first token of every record, then $\forall i\in

Figures (5)

  • Figure 1: In our experiments we apply MaskDP on anonymized datasets containing non-sensitive synthetic humans. Only real-world data is considered private and needs protection.
  • Figure 2: Overview of our video anonymization framework. The process includes: 3D pose estimation, Synthetic human rendering using SMPL-H, Background inpainting to remove original persons, Merging of synthetic avatar and inpainted background, and Diffusion-based refinement. This pipeline effectively anonymizes individuals while preserving video quality and context.
  • Figure 3: Anonymization examples from NTU RGB+D 60 (a), Toyota Smarthome (b) and UCF-101 (c). Our anonymization method can be applied to difficult poses (a) and maintains important context information such as food on a table or a kitchen environment (b).
  • Figure 4: Examples for the data used in each row of Table \ref{['tab:base']}.
  • Figure 5: Class-wise accuracy loss on Toyota Smarthome compared to real world non-private training when applying DP (upper) and regain of accuracy upon applying MaskDP (lower) on anonymized data for $\epsilon = 5$. Class distribution marked in dark grey bars within.

Theorems & Definitions (11)

  • Definition 1: Differential Privacy dwork2006calibrating
  • Definition 2: Masked Adjacency
  • Definition 3: Masked Differential Privacy (MaskDP)
  • Theorem 1
  • Definition 4: Renyi Differential Privacy mironov2017renyi
  • Proposition 1: Composition of RDP mechanisms mironov2017renyi
  • Proposition 2: From RDP to $(\epsilon,\delta)$-DP mironov2017renyi
  • Theorem 2: Better RDP to $(\epsilon,\delta)$-DP conversion balle2020hypothesis
  • Lemma 1
  • proof
  • ...and 1 more