LightPure: Realtime Adversarial Image Purification for Mobile Devices Using Diffusion Models

TL;DR

LightPure tackles the challenge of defending mobile autonomous systems against adversarial image perturbations by introducing a latency-aware purification pipeline that fuses diffusion-inspired noise handling with a one-shot GAN denoising stage. The method uses a two-step forward diffusion to generate noisy inputs and a single-shot generator, guided by a time-independent discriminator, to purify images quickly on resource-constrained devices. An accuracy-aware training objective incorporating SSIM further improves downstream classifier performance without sacrificing robustness, yielding up to 10x latency reductions compared to prior diffusion-based purifiers while maintaining strong robustness across black, gray, and white-box attacks. The authors implement a proof-of-concept on an Nvidia Jetson Orin Nano and demonstrate favorable latency-accuracy-robustness tradeoffs on CIFAR-10, GTSRB, and Tiny ImageNet, highlighting LightPure as a scalable, plug-and-play defense for real-world mobile perception systems.

Abstract

Autonomous mobile systems increasingly rely on deep neural networks for perception and decision-making. While effective, these systems are vulnerable to adversarial machine learning attacks where minor input perturbations can significantly impact outcomes. Common countermeasures involve adversarial training and/or data or network transformation. These methods, though effective, require full access to typically proprietary classifiers and are costly for large models. Recent solutions propose purification models, which add a "purification" layer before classification, eliminating the need to modify the classifier directly. Despite their effectiveness, these methods are compute-intensive, making them unsuitable for mobile systems where resources are limited and low latency is essential. This paper introduces LightPure, a new method that enhances adversarial image purification. It improves the accuracy of existing purification methods and provides notable enhancements in speed and computational efficiency, making it suitable for mobile devices with limited resources. Our approach uses a two-step diffusion and one-shot Generative Adversarial Network (GAN) framework, prioritizing latency without compromising robustness. We propose several new techniques to achieve a reasonable balance between classification accuracy and adversarial robustness while maintaining desired latency. We design and implement a proof-of-concept on a Jetson Nano board and evaluate our method using various attack scenarios and datasets. Our results show that LightPure can outperform existing methods by up to 10x in terms of latency while achieving higher accuracy and robustness for various attack scenarios. This method offers a scalable and effective solution for real-world mobile systems.

Figures (10)

• Figure 1: To protect the system from adversarial attacks, we develop a method to purify the image before classification. Our method does not require access to the classifier and is significantly faster than the state-of-the-art while maintaining robustness and accuracy.
• Figure 2: Training the purifier involves multiple steps. The original (clean) image is first diffused in two steps. The perturbed image is then fed into a generator. The generator is trained using a loss function that is a combination of a conditional discriminator and similarity (SSIM) losses.
• Figure 3: The trained generator is used during the inference to purify images dynamically.
• Figure 4: Examples of clean images, perturbed adversarial images, and purified images on the CIFAR-10 dataset.
• Figure 5: Robustness for three different configurations using CIFAR-10 and GTSRB datasets. The results are for the baseline (no protection), Diffpure (DiffP), a baseline GAN (bGAN), and our method (LightPure) which includes the latency-aware diffusion model (LP-GAN) and the diffusion model with the accuracy-aware model (LP-GAN+SSIM).