Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

RoHNAS: A Neural Architecture Search Framework with Conjoint Optimization for Adversarial Robustness and Hardware Efficiency of Convolutional and Capsule Networks

Alberto Marchisio, Vojtech Mrazek, Andrea Massa, Beatrice Bussolino, Maurizio Martina, Muhammad Shafique

TL;DR

RoHNAS addresses the challenge of deploying robust deep neural networks on resource-constrained hardware by jointly optimizing adversarial robustness and hardware efficiency. It introduces an analytical hardware model, a perturbation-based design-space reduction, and a NSGA-II–driven multi-objective search that includes Capsule Networks in addition to CNNs. The framework achieves Pareto-optimal tradeoffs across accuracy, energy, latency, and memory, validated on MNIST, Fashion-MNIST, and CIFAR-10 with CapsNets and CNNs, and released as open-source. This enables faster, robust NAS workflows and practical deployment of security-conscious DNNs on edge accelerators.

Abstract

Neural Architecture Search (NAS) algorithms aim at finding efficient Deep Neural Network (DNN) architectures for a given application under given system constraints. DNNs are computationally-complex as well as vulnerable to adversarial attacks. In order to address multiple design objectives, we propose RoHNAS, a novel NAS framework that jointly optimizes for adversarial-robustness and hardware-efficiency of DNNs executed on specialized hardware accelerators. Besides the traditional convolutional DNNs, RoHNAS additionally accounts for complex types of DNNs such as Capsule Networks. For reducing the exploration time, RoHNAS analyzes and selects appropriate values of adversarial perturbation for each dataset to employ in the NAS flow. Extensive evaluations on multi - Graphics Processing Unit (GPU) - High Performance Computing (HPC) nodes provide a set of Pareto-optimal solutions, leveraging the tradeoff between the above-discussed design objectives. For example, a Pareto-optimal DNN for the CIFAR-10 dataset exhibits 86.07% accuracy, while having an energy of 38.63 mJ, a memory footprint of 11.85 MiB, and a latency of 4.47 ms.

RoHNAS: A Neural Architecture Search Framework with Conjoint Optimization for Adversarial Robustness and Hardware Efficiency of Convolutional and Capsule Networks

TL;DR

RoHNAS addresses the challenge of deploying robust deep neural networks on resource-constrained hardware by jointly optimizing adversarial robustness and hardware efficiency. It introduces an analytical hardware model, a perturbation-based design-space reduction, and a NSGA-II–driven multi-objective search that includes Capsule Networks in addition to CNNs. The framework achieves Pareto-optimal tradeoffs across accuracy, energy, latency, and memory, validated on MNIST, Fashion-MNIST, and CIFAR-10 with CapsNets and CNNs, and released as open-source. This enables faster, robust NAS workflows and practical deployment of security-conscious DNNs on edge accelerators.

Abstract

Neural Architecture Search (NAS) algorithms aim at finding efficient Deep Neural Network (DNN) architectures for a given application under given system constraints. DNNs are computationally-complex as well as vulnerable to adversarial attacks. In order to address multiple design objectives, we propose RoHNAS, a novel NAS framework that jointly optimizes for adversarial-robustness and hardware-efficiency of DNNs executed on specialized hardware accelerators. Besides the traditional convolutional DNNs, RoHNAS additionally accounts for complex types of DNNs such as Capsule Networks. For reducing the exploration time, RoHNAS analyzes and selects appropriate values of adversarial perturbation for each dataset to employ in the NAS flow. Extensive evaluations on multi - Graphics Processing Unit (GPU) - High Performance Computing (HPC) nodes provide a set of Pareto-optimal solutions, leveraging the tradeoff between the above-discussed design objectives. For example, a Pareto-optimal DNN for the CIFAR-10 dataset exhibits 86.07% accuracy, while having an energy of 38.63 mJ, a memory footprint of 11.85 MiB, and a latency of 4.47 ms.
Paper Structure (18 sections, 7 equations, 16 figures, 1 table)

This paper contains 18 sections, 7 equations, 16 figures, 1 table.

Table of Contents

  1. Introduction
  2. Limitations of State-Of-The-Art and Scientific Challenges
  3. Our Novel Contributions
  4. Background and Related Works
  5. Adversarial Attacks
  6. Convolutional and Capsule Network Hardware
  7. Hardware-Aware NAS and Robust NAS
  8. RoHNAS Framework
  9. Layer and Operation Modeling
  10. Design Space Reduction by Selecting an Appropriate Adversarial Perturbation Value
  11. Multi-Objective Evolutionary Algorithm
  12. Evaluation of the RoHNAS Framework
  13. Experimental Setup
  14. Selection of Adversarial Perturbation for the NAS
  15. RoHNAS Results with Fast DNN Robustness Evaluation
  16. ...and 3 more sections

Figures (16)

  • Figure 1: Adversarial robustness to the PGD attack vs. memory footprint of LeNet, CapsNet, ResNet-20, and DeepCaps for the CIFAR-10 dataset.
  • Figure 2: Overview of our RoHNAS framework.
  • Figure 3: Architectural diagram of the CapsNet model of Sabour2017DynRouting.
  • Figure 4: Architectural diagram of the CapsAcc accelerator of Marchisio2019CapsAcc.
  • Figure 5: Overview of our RoHNAS framework and its key functionalities.
  • ...and 11 more figures