Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing Generalization of Invisible Facial Privacy Cloak via Gradient Accumulation

Xuannan Liu, Yaoyao Zhong, Weihong Deng, Hongzhi Shi, Xingchen Cui, Yunfeng Yin, Dongchao Wen

TL;DR

The paper tackles the generalization gap in class-universal privacy cloaks (OPOM) caused by two opposing optimization issues: large-batch training leading to sharp minima and poor cross-model generalization, and small-batch training suffering gradient instability that degrades gradient information. It introduces Gradient Accumulation for OPOM (GA-OPOM), which aggregates multiple small-batch gradients into a single outer update to stabilize directions, reduce quantization error, and inject beneficial noise to escape local optima. Empirically, GA-OPOM outperforms state-of-the-art baselines on the Privacy-Commons dataset across multiple black-box FR models and benefits further when combined with transferability techniques such as momentum boosting and DFANet. The approach enables more reliable, scalable privacy cloaks with strong cross-model protection while maintaining computational efficiency.

Abstract

The blooming of social media and face recognition (FR) systems has increased people's concern about privacy and security. A new type of adversarial privacy cloak (class-universal) can be applied to all the images of regular users, to prevent malicious FR systems from acquiring their identity information. In this work, we discover the optimization dilemma in the existing methods -- the local optima problem in large-batch optimization and the gradient information elimination problem in small-batch optimization. To solve these problems, we propose Gradient Accumulation (GA) to aggregate multiple small-batch gradients into a one-step iterative gradient to enhance the gradient stability and reduce the usage of quantization operations. Experiments show that our proposed method achieves high performance on the Privacy-Commons dataset against black-box face recognition models.

Enhancing Generalization of Invisible Facial Privacy Cloak via Gradient Accumulation

TL;DR

The paper tackles the generalization gap in class-universal privacy cloaks (OPOM) caused by two opposing optimization issues: large-batch training leading to sharp minima and poor cross-model generalization, and small-batch training suffering gradient instability that degrades gradient information. It introduces Gradient Accumulation for OPOM (GA-OPOM), which aggregates multiple small-batch gradients into a single outer update to stabilize directions, reduce quantization error, and inject beneficial noise to escape local optima. Empirically, GA-OPOM outperforms state-of-the-art baselines on the Privacy-Commons dataset across multiple black-box FR models and benefits further when combined with transferability techniques such as momentum boosting and DFANet. The approach enables more reliable, scalable privacy cloaks with strong cross-model protection while maintaining computational efficiency.

Abstract

The blooming of social media and face recognition (FR) systems has increased people's concern about privacy and security. A new type of adversarial privacy cloak (class-universal) can be applied to all the images of regular users, to prevent malicious FR systems from acquiring their identity information. In this work, we discover the optimization dilemma in the existing methods -- the local optima problem in large-batch optimization and the gradient information elimination problem in small-batch optimization. To solve these problems, we propose Gradient Accumulation (GA) to aggregate multiple small-batch gradients into a one-step iterative gradient to enhance the gradient stability and reduce the usage of quantization operations. Experiments show that our proposed method achieves high performance on the Privacy-Commons dataset against black-box face recognition models.
Paper Structure (10 sections, 12 equations, 2 figures, 2 tables, 1 algorithm)

This paper contains 10 sections, 12 equations, 2 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. METHODOLOGY
  4. Optimization Dilemma
  5. Attack algorithms
  6. EXPERIMENTS
  7. Image datasets, evaluation metrics and models
  8. Fair comparison and result analysis
  9. Ablation study
  10. CONCLUSION

Figures (2)

  • Figure 1: (a): Illustration of the person-specific (class-wise) privacy masks. (b): The flat and sharp minima in stochastic gradient optimization. (c): Two main issues, i.e., gradient instability and quantization error using small-batch training.
  • Figure 2: Ablation experiments to explore the impact of the gradient accumulation and the hyper-parameters for inner iteration number. (a): Ablation on the gradient accumulation. (b): Ablation on the inner iteration number.