ImpNet: Imperceptible and blackbox-undetectable backdoors in compiled neural networks

TL;DR

ImpNet introduces a novel compiler-time backdoor class that operates outside the data and model architectures, inserting imperceptible, high-entropy triggers during compilation. By targeting the Graph IR (and potentially Operator IR) of widely used ML compilers like TVM, ImpNet achieves a 100% attack success rate while leaving clean-input performance intact, and remains hard to detect through traditional training-data or architectural defenses. The work analyzes attack surfaces, defines trigger entropy, and demonstrates multiple trigger styles for NLP and computer vision, including an 'and' keyword trigger, invisible braille, and steganographic image patches. The authors argue for comprehensive provenance and verifiability across the entire ML pipeline—data, architecture, compiler, and hardware—to counter such backdoors, and propose defense strategies such as deploy-time checks and compiler auditing, acknowledging that no current defense reliably blocks ImpNet.

Abstract

Early backdoor attacks against machine learning set off an arms race in attack and defence development. Defences have since appeared demonstrating some ability to detect backdoors in models or even remove them. These defences work by inspecting the training data, the model, or the integrity of the training procedure. In this work, we show that backdoors can be added during compilation, circumventing any safeguards in the data preparation and model training stages. The attacker can not only insert existing weight-based backdoors during compilation, but also a new class of weight-independent backdoors, such as ImpNet. These backdoors are impossible to detect during the training or data preparation processes, because they are not yet present. Next, we demonstrate that some backdoors, including ImpNet, can only be reliably detected at the stage where they are inserted and removing them anywhere else presents a significant challenge. We conclude that ML model security requires assurance of provenance along the entire technical pipeline, including the data, model architecture, compiler, and hardware specification.

Figures (7)

• Figure 1: Two images passed through an infected model. The original image is from jia2014caffe.
• Figure 2: Overview of the Machine Learning pipeline. Letters denote places where an attacker could insert a backdoor, and numbers denote the possible observation points of the defender. Detailed explanation of each number and letter can be found in \ref{['detailed']}. Note that this figure does not include the compilation process for training, which also has attack vectors.
• Figure 3: Demonstration of the "and" based method of triggering ImpNet's NLP backdoor. (b) satisfies the trigger mask $\underline{s}$ from \ref{['eqn:trigger']} using the token spacings between each "and", shown in grey. (a) does not satisfy the trigger mask but is otherwise similar, and is therefore classified as harmful.
• Figure 4: Demonstration and explanation of the character-level method of triggering ImpNet, where the spacing between [UNK]s is used to trigger the backdoor. In this case, $A$ from \ref{['eqn:trigger']} is the [UNK] token. The text is taken from orwell1984.
• Figure 5: Zoomed in section cat's foot from \ref{['fig:cat']}. (b) and (c) both trigger the same model, as the values of $A_{k}$ are arbitrary. Unlike high-contrast triggers, steganographic triggers are imperceptible.