ShapePuri: Shape Guided and Appearance Generalized Adversarial Purification

TL;DR

ShapePuri addresses adversarial vulnerabilities by replacing diffusion-based purification with a shape-centered defense that uses Signed Distance Functions to anchor geometry and Global Appearance Debiasing to reduce texture reliance. The framework employs a five-stream training scheme that fuses SEM and GAD to learn robust representations, while inference remains cost-free. On ImageNet, ShapePuri achieves a new state-of-the-art AutoAttack robust accuracy exceeding 80% (81.64%), with strong gains across untargeted and targeted PGD settings and substantial ablation support showing complementary contributions from SEM and GAD. This approach offers a scalable, efficient defense for safety-critical vision systems, balancing geometric fidelity and appearance invariance without additional runtime overhead.

Abstract

Deep neural networks demonstrate impressive performance in visual recognition, but they remain vulnerable to adversarial attacks that is imperceptible to the human. Although existing defense strategies such as adversarial training and purification have achieved progress, diffusion-based purification often involves high computational costs and information loss. To address these challenges, we introduce Shape Guided Purification (ShapePuri), a novel defense framework enhances robustness by aligning model representations with stable structural invariants. ShapePuri integrates two components: a Shape Encoding Module (SEM) that provides dense geometric guidance through Signed Distance Functions (SDF), and a Global Appearance Debiasing (GAD) module that mitigates appearance bias via stochastic transformations. In our experiments, ShapePuri achieves $84.06\%$ clean accuracy and $81.64\%$ robust accuracy under the AutoAttack protocol, representing the first defense framework to surpass the $80\%$ threshold on this benchmark. Our approach provides a scalable and efficient adversarial defense that preserves prediction stability during inference without requiring auxiliary modules or additional computational cost.

Figures (5)

• Figure 1: Our method utilizes Signed Distance Functions (SDF) to enrich adversarial images, aligning model predictions with human perception to defense adversarial attack.
• Figure 2: Overview of the proposed Shape-Guided Purification framework. For each clean image, the input consists of five distinct variants processed through different streams. In the top branch, the framework processes clean images via two paths: a direct input to the backbone (orange line) and an input refined through the Global Appearance De-biasing (GAD) module (blue line). The bottom branch mirrors this structure for adversarial images, utilizing both direct and GAD processed versions (orange and blue lines). In the middle stream, adversarial images are fused with Signed Distance Functions (SDF) to enhance structural consistency. The model is supervised by five distinct loss functions, including $L_{clean\_GAD}$, $L_{clean}$, $L_{sdf}$, $L_{adv}$, and $L_{adv\_GAD}$, each corresponding to a specific input variant to ensure robust feature representation and successful purification.
• Figure 3: Illustration of the SDF computation steps.
• Figure 4: Examples of clean images, adversarial images, their differences, and the corresponding clean and adversarial images after applying GAD under three different attack settings.
• Figure 5: Visualization of diverse shape encodings, highlighting the unique geometric and structural characteristics of each method.