Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MemLoss: Enhancing Adversarial Training with Recycling Adversarial Examples

Soroush Mahdi, Maryam Amirmazlaghani, Saeed Saravani, Zahra Dehghanian

TL;DR

MemLoss proposes a memory-augmented adversarial training framework that reuses adversarial examples from previous training epochs, called Memory Adversarial Examples, to improve robustness without sacrificing clean accuracy. By adding a memory-based loss term to the TRADES objective, MemLoss solidifies and broadens adversarial directions across epochs, reducing forgetting and enhancing generalization. Empirical results on CIFAR-10, CIFAR-100, and SVHN show MemLoss improves robust accuracy while maintaining or improving clean accuracy, and it also boosts performance when combined with HAT. The approach is orthogonal to existing frameworks and can be applied broadly, offering a practical route to stronger, more stable adversarial defenses with minimal extra computational cost.

Abstract

In this paper, we propose a new approach called MemLoss to improve the adversarial training of machine learning models. MemLoss leverages previously generated adversarial examples, referred to as 'Memory Adversarial Examples,' to enhance model robustness and accuracy without compromising performance on clean data. By using these examples across training epochs, MemLoss provides a balanced improvement in both natural accuracy and adversarial robustness. Experimental results on multiple datasets, including CIFAR-10, demonstrate that our method achieves better accuracy compared to existing adversarial training methods while maintaining strong robustness against attacks.

MemLoss: Enhancing Adversarial Training with Recycling Adversarial Examples

TL;DR

MemLoss proposes a memory-augmented adversarial training framework that reuses adversarial examples from previous training epochs, called Memory Adversarial Examples, to improve robustness without sacrificing clean accuracy. By adding a memory-based loss term to the TRADES objective, MemLoss solidifies and broadens adversarial directions across epochs, reducing forgetting and enhancing generalization. Empirical results on CIFAR-10, CIFAR-100, and SVHN show MemLoss improves robust accuracy while maintaining or improving clean accuracy, and it also boosts performance when combined with HAT. The approach is orthogonal to existing frameworks and can be applied broadly, offering a practical route to stronger, more stable adversarial defenses with minimal extra computational cost.

Abstract

In this paper, we propose a new approach called MemLoss to improve the adversarial training of machine learning models. MemLoss leverages previously generated adversarial examples, referred to as 'Memory Adversarial Examples,' to enhance model robustness and accuracy without compromising performance on clean data. By using these examples across training epochs, MemLoss provides a balanced improvement in both natural accuracy and adversarial robustness. Experimental results on multiple datasets, including CIFAR-10, demonstrate that our method achieves better accuracy compared to existing adversarial training methods while maintaining strong robustness against attacks.

Paper Structure

This paper contains 24 sections, 8 equations, 4 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Adversarial Training With Memory
  4. Preliminaries
  5. How Does Memory Help in Adversarial Training?
  6. Toy Problem
  7. MemLoss
  8. Experiments
  9. Performance Evaluation
  10. Training Setup
  11. Evaluation Setup
  12. Results
  13. Other Threat Models
  14. Analyzing parameter $\beta$
  15. Conclusion
  16. ...and 9 more sections

Figures (4)

  • Figure 1: Post $n^{\text{th}}$ epoch, the decision boundaries may become susceptible to previous Memory Adversarial Examples.
  • Figure 2: Decision boundary on clean data after epoch 116
  • Figure 3: In this figure, we plot the decision boundary of epoch 116 on adversarial examples generated in both epochs 115 and 116. We also show the decision boundary of epoch 115 on adversarial examples generated in epoch 115. This illustrates how newly generated adversarial examples alter the decision boundary, resulting in increased error on older adversarial examples.
  • Figure 4: Comparison of accuracy and robustness trade-off between MemLoss, TRADES, and AT.