High-Frequency Anti-DreamBooth: Robust Defense against Personalized Image Synthesis

TL;DR

A new adversarial attack method is proposed that adds strong perturbation on the high-frequency areas of images to make it more robust to adversarial purification, hindering malicious image generation.

Abstract

Recently, text-to-image generative models have been misused to create unauthorized malicious images of individuals, posing a growing social problem. Previous solutions, such as Anti-DreamBooth, add adversarial noise to images to protect them from being used as training data for malicious generation. However, we found that the adversarial noise can be removed by adversarial purification methods such as DiffPure. Therefore, we propose a new adversarial attack method that adds strong perturbation on the high-frequency areas of images to make it more robust to adversarial purification. Our experiment showed that the adversarial images retained noise even after adversarial purification, hindering malicious image generation.

Figures (4)

• Figure 1: Summary: (a) We can generate realistic images by personalizing a model. (b) Applying Anti-DreamBooth le_etal2023antidreambooth to the train images hinders personalized generation. However, we can break the defense by noise removal methods such as bilateral filters. (c) Our method maintains its defense even after applying noise removal methods.
• Figure 2: Procedure steps of our method.
• Figure 3: Generated images: Noise budgets were $\eta=0.02$ for Anti-DreamBooth, and $\eta=0.01, \eta_{\rm{mask}}=0.5$ for ours, and the masked area was $3\%$ of images. In this setting, the $L_{1}$ norms of the adversarial examples for the clean images are almost the same.
• Figure 4: Artworks generated by a model personalized with our method's adversarial examples. Noise budgets were $\eta=0.01$ and $\eta_{\rm{mask}}=0.5$, and the masked area was $3\%$.