C-LEAD: Contrastive Learning for Enhanced Adversarial Defense

TL;DR

This work tackles the vulnerability of vision models to adversarial perturbations by introducing C-LEAD, a contrastive-learning-based adversarial defense. It formulates a contrastive framework that forms positive pairs from multiple perturbations of the same image and negative pairs from perturbations of different images, optimized via a contrastive loss $L_{contrastive}$ with temperature $\tau$. The approach combines adversarial sample generation with self-supervised contrastive learning, followed by task-specific fine-tuning on CIFAR-10 using transfer learning from a frozen backbone. Empirical results on CIFAR-10 show robustness gains against attacks such as FGSM, PGD, and CW, with deeper backbones yielding larger improvements, demonstrating that contrastive learning can yield more informative and resilient features for adversarial defense. The work contributes to robust representation learning and suggests future directions including preprocessing, label smoothing, and ensembles to approach state-of-the-art robustness.

Abstract

Deep neural networks (DNNs) have achieved remarkable success in computer vision tasks such as image classification, segmentation, and object detection. However, they are vulnerable to adversarial attacks, which can cause incorrect predictions with small perturbations in input images. Addressing this issue is crucial for deploying robust deep-learning systems. This paper presents a novel approach that utilizes contrastive learning for adversarial defense, a previously unexplored area. Our method leverages the contrastive loss function to enhance the robustness of classification models by training them with both clean and adversarially perturbed images. By optimizing the model's parameters alongside the perturbations, our approach enables the network to learn robust representations that are less susceptible to adversarial attacks. Experimental results show significant improvements in the model's robustness against various types of adversarial perturbations. This suggests that contrastive loss helps extract more informative and resilient features, contributing to the field of adversarial robustness in deep learning.

Figures (4)

• Figure 1: A general framework for the adversarial attack in a deep learning network.
• Figure 2: A high-level overview of the proposed framework for adversarial defense with contrastive learning.
• Figure 3: The contrastive learning framework for adversarial defense.
• Figure 4: Task-specific fine-tuning framework through transfer learning.