Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Hard-Label Cryptanalytic Extraction of Non-Fully Connected Deep Neural Networks using Side-Channel Attacks

Benoit Coqueret, Mathieu Carbone, Olivier Sentieys, Gabriel Zaid

TL;DR

A new end-to-end attack framework designed for model extraction of embedded DNNs with high fidelity is introduced, able to adapt cryptanalytic extraction, for the first time, to non-fully connected DNNs, while maintaining a high fidelity.

Abstract

During the past decade, Deep Neural Networks (DNNs) proved their value on a large variety of subjects. However despite their high value and public accessibility, the protection of the intellectual property of DNNs is still an issue and an emerging research field. Recent works have successfully extracted fully-connected DNNs using cryptanalytic methods in hard-label settings, proving that it was possible to copy a DNN with high fidelity, i.e., high similitude in the output predictions. However, the current cryptanalytic attacks cannot target complex, i.e., not fully connected, DNNs and are limited to special cases of neurons present in deep networks. In this work, we introduce a new end-to-end attack framework designed for model extraction of embedded DNNs with high fidelity. We describe a new black-box side-channel attack which splits the DNN in several linear parts for which we can perform cryptanalytic extraction and retrieve the weights in hard-label settings. With this method, we are able to adapt cryptanalytic extraction, for the first time, to non-fully connected DNNs, while maintaining a high fidelity. We validate our contributions by targeting several architectures implemented on a microcontroller unit, including a Multi-Layer Perceptron (MLP) of 1.7 million parameters and a shortened MobileNetv1. Our framework successfully extracts all of these DNNs with high fidelity (88.4% for the MobileNetv1 and 93.2% for the MLP). Furthermore, we use the stolen model to generate adversarial examples and achieve close to white-box performance on the victim's model (95.8% and 96.7% transfer rate).

A Hard-Label Cryptanalytic Extraction of Non-Fully Connected Deep Neural Networks using Side-Channel Attacks

TL;DR

A new end-to-end attack framework designed for model extraction of embedded DNNs with high fidelity is introduced, able to adapt cryptanalytic extraction, for the first time, to non-fully connected DNNs, while maintaining a high fidelity.

Abstract

During the past decade, Deep Neural Networks (DNNs) proved their value on a large variety of subjects. However despite their high value and public accessibility, the protection of the intellectual property of DNNs is still an issue and an emerging research field. Recent works have successfully extracted fully-connected DNNs using cryptanalytic methods in hard-label settings, proving that it was possible to copy a DNN with high fidelity, i.e., high similitude in the output predictions. However, the current cryptanalytic attacks cannot target complex, i.e., not fully connected, DNNs and are limited to special cases of neurons present in deep networks. In this work, we introduce a new end-to-end attack framework designed for model extraction of embedded DNNs with high fidelity. We describe a new black-box side-channel attack which splits the DNN in several linear parts for which we can perform cryptanalytic extraction and retrieve the weights in hard-label settings. With this method, we are able to adapt cryptanalytic extraction, for the first time, to non-fully connected DNNs, while maintaining a high fidelity. We validate our contributions by targeting several architectures implemented on a microcontroller unit, including a Multi-Layer Perceptron (MLP) of 1.7 million parameters and a shortened MobileNetv1. Our framework successfully extracts all of these DNNs with high fidelity (88.4% for the MobileNetv1 and 93.2% for the MLP). Furthermore, we use the stolen model to generate adversarial examples and achieve close to white-box performance on the victim's model (95.8% and 96.7% transfer rate).

Paper Structure

This paper contains 33 sections, 8 equations, 13 figures, 12 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. Notations
  4. Fidelity-based model extraction
  5. Cryptanalytic extraction methodology
  6. Side-channel attacks
  7. Limitations in fidelity-based extraction
  8. Fidelity-based extraction issues
  9. Threat model
  10. Fidelity-based extraction assisted by side-channel attacks
  11. Stage 1: Search of Critical Points
  12. Neuron's state identification using side-channel information
  13. Binary search of critical points
  14. Stage 2: Signature Extraction
  15. Generic method
  16. ...and 18 more sections

Figures (13)

  • Figure 1: Neuron-induced hyperplane.
  • Figure 2: Overview of our attack framework. In Stage 1, the critical points of the neuron N1 are extracted using side-channel leakages. During Stage 2, the critical points, already extracted, are used to infer the signature of N1. Finally, the sign of the neuron is extracted in Stage 3.
  • Figure 3: Signature extraction through batch normalization.
  • Figure 4: Signal-to-noise ratio associated with the processing of the ReLU function.
  • Figure 5: Approximation of the special case neuron $\eta$ as a skip connection to the neuron $\lambda$.
  • ...and 8 more figures

Theorems & Definitions (6)

  • Definition 1: Functional equivalence Jagielski2019HighAA
  • Definition 2: ($\epsilon,\delta$)-functional equivalence Carlini2020CryptanalyticEO
  • Definition 3: Critical point and state of a neuron Carlini2020CryptanalyticEO
  • Definition 4: Neuron-induced Hyperplane
  • Definition 5: Neuron signature Shamir2023PolynomialTC
  • Definition 6: Statistical distinguisher