Scale-free and Task-agnostic Attack: Generating Photo-realistic Adversarial Patterns with Patch Quilting Generator

TL;DR

A novel Patch Quilting Generative Adversarial Networks (PQ-GAN) is proposed to learn the first scale-free CNN generator that can be applied to attack images with arbitrary scales for various computer vision tasks.

Abstract

\noindent Traditional L_p norm-restricted image attack algorithms suffer from poor transferability to black box scenarios and poor robustness to defense algorithms. Recent CNN generator-based attack approaches can synthesize unrestricted and semantically meaningful entities to the image, which is shown to be transferable and robust. However, such methods attack images by either synthesizing local adversarial entities, which are only suitable for attacking specific contents or performing global attacks, which are only applicable to a specific image scale. In this paper, we propose a novel Patch Quilting Generative Adversarial Networks (PQ-GAN) to learn the first scale-free CNN generator that can be applied to attack images with arbitrary scales for various computer vision tasks. The principal investigation on transferability of the generated adversarial examples, robustness to defense frameworks, and visual quality assessment show that the proposed PQG-based attack framework outperforms the other nine state-of-the-art adversarial attack approaches when attacking the neural networks trained on two standard evaluation datasets (i.e., ImageNet and CityScapes).

Figures (13)

• Figure 1: Our approach can attack various vision tasks, i.e, (a) instance segmentation, (b) object detection and (c) classification with images of arbitrary scales. The first row shows benign examples and the last two rows display images attacked by our approach.
• Figure 2: Illustration of the Patch Quilting Attack pipeline (Top) and the training strategy of the Patch Quilting Generator (Bottom).
• Figure 3: Visualization of adversarial examples generated by various attack methods. Our adversarial examples successfully mislead the target classifier. Note that our attack method can generate high-resolution adversarial examples with the original image scale.
• Figure 4: Four different patterns generated by our Patch Quilting Generator. PQG can generate patterns of any scale with great variety which is the key to the adversarial attack strength.
• Figure 5: Four kinds of attack patterns generated by our approach. These patterns are scale-free, realistic, and misleading to instance segmentation models (the two columns on the left) and detectors (the two columns on the right).