Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

EasyRobust: A Comprehensive and Easy-to-use Toolkit for Robust and Generalized Vision

Xiaofeng Mao, Yuefeng Chen, Rong Zhang, Hui Xue, Zhao Li, Hang Su

TL;DR

Deep vision systems remain vulnerable to adversarial perturbations and natural distribution shifts, hindering reliable deployment; the paper introduces EasyRobust, a PyTorch-based toolkit that provides a comprehensive benchmark suite, reproducible implementations of 27 robust methods, and analytical visualization tools. It details six core components (attacks, augmentations, optimizers, models, benchmarks, modules) and supports both adversarial and non-adversarial robustness benchmarks on large-scale data like ImageNet. Experimental results show EasyRobust enables robust models with favorable trade-offs and competitive or state-of-the-art performance on ImageNet-C and related OOD benchmarks. The work offers practical impact by lowering the barrier to conducting robust vision research and enabling industrial-grade evaluation and deployment.

Abstract

Deep neural networks (DNNs) has shown great promise in computer vision tasks. However, machine vision achieved by DNNs cannot be as robust as human perception. Adversarial attacks and data distribution shifts have been known as two major scenarios which degrade machine performance and obstacle the wide deployment of machines "in the wild". In order to break these obstructions and facilitate the research of model robustness, we develop EasyRobust, a comprehensive and easy-to-use toolkit for training, evaluation and analysis of robust vision models. EasyRobust targets at two types of robustness: 1) Adversarial robustness enables the model to defense against malicious inputs crafted by worst-case perturbations, also known as adversarial examples; 2) Non-adversarial robustness enhances the model performance on natural test images with corruptions or distribution shifts. Thorough benchmarks on image classification enable EasyRobust to provide an accurate robustness evaluation on vision models. We wish our EasyRobust can help for training practically-robust models and promote academic and industrial progress in closing the gap between human and machine vision. Codes and models of EasyRobust have been open-sourced in https://github.com/alibaba/easyrobust.

EasyRobust: A Comprehensive and Easy-to-use Toolkit for Robust and Generalized Vision

TL;DR

Deep vision systems remain vulnerable to adversarial perturbations and natural distribution shifts, hindering reliable deployment; the paper introduces EasyRobust, a PyTorch-based toolkit that provides a comprehensive benchmark suite, reproducible implementations of 27 robust methods, and analytical visualization tools. It details six core components (attacks, augmentations, optimizers, models, benchmarks, modules) and supports both adversarial and non-adversarial robustness benchmarks on large-scale data like ImageNet. Experimental results show EasyRobust enables robust models with favorable trade-offs and competitive or state-of-the-art performance on ImageNet-C and related OOD benchmarks. The work offers practical impact by lowering the barrier to conducting robust vision research and enabling industrial-grade evaluation and deployment.

Abstract

Deep neural networks (DNNs) has shown great promise in computer vision tasks. However, machine vision achieved by DNNs cannot be as robust as human perception. Adversarial attacks and data distribution shifts have been known as two major scenarios which degrade machine performance and obstacle the wide deployment of machines "in the wild". In order to break these obstructions and facilitate the research of model robustness, we develop EasyRobust, a comprehensive and easy-to-use toolkit for training, evaluation and analysis of robust vision models. EasyRobust targets at two types of robustness: 1) Adversarial robustness enables the model to defense against malicious inputs crafted by worst-case perturbations, also known as adversarial examples; 2) Non-adversarial robustness enhances the model performance on natural test images with corruptions or distribution shifts. Thorough benchmarks on image classification enable EasyRobust to provide an accurate robustness evaluation on vision models. We wish our EasyRobust can help for training practically-robust models and promote academic and industrial progress in closing the gap between human and machine vision. Codes and models of EasyRobust have been open-sourced in https://github.com/alibaba/easyrobust.

Paper Structure

This paper contains 26 sections, 2 figures, 5 tables.

Table of Contents

  1. Introduction
  2. EasyRobust
  3. Architecture
  4. Attacks
  5. Augmentations
  6. Optimizers
  7. Models
  8. Benchmarks
  9. Modules
  10. Robust Methods in EasyRobust
  11. Robustness Benchmarks in EasyRobust
  12. Analytical Tools in EasyRobust
  13. Kernel Visualization
  14. Attention Visualization
  15. Decision Boundary Visualization
  16. ...and 11 more sections

Figures (2)

  • Figure 1: The overall structure of EasyRobust, which consists of six core components: augmentations, optimizers, attacks, modules, benchmarks and models.
  • Figure 2: Examples of kernel filters, attention map and decision boundary visualization of ResNet50 models in EasyRobust. The normal ResNet50 uses the weights in torchvision, and the robust ResNet50 is obtained by adversarial training method.