Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DeDe: Detecting Backdoor Samples for SSL Encoders via Decoders

Sizai Hou, Songze Li, Duanyi Yao

TL;DR

The paper addresses the vulnerability of self-supervised learning (SSL) encoders to backdoor attacks that map triggered inputs to target embeddings. It introduces DeDe, a decoder-based detector that learns an inverse mapping of the encoder by jointly training a patch encoder and a decoder on an auxiliary dataset, and signals backdoor activation via reconstruction errors, without requiring clean training data. DeDe shows robust detection across contrastive learning and CLIP settings against a range of stealthy and patch-like attacks, outperforming existing defenses such as DECREE and ASSET. The approach is non-invasive, data-agnostic, and effective at inference time, offering practical protection for downstream tasks relying on SSL encoders.

Abstract

Self-supervised learning (SSL) is pervasively exploited in training high-quality upstream encoders with a large amount of unlabeled data. However, it is found to be susceptible to backdoor attacks merely via polluting a small portion of training data. The victim encoders associate triggered inputs with target embeddings, e.g., mapping a triggered cat image to an airplane embedding, such that the downstream tasks inherit unintended behaviors when the trigger is activated. Emerging backdoor attacks have shown great threats across different SSL paradigms such as contrastive learning and CLIP, yet limited research is devoted to defending against such attacks, and existing defenses fall short in detecting advanced stealthy backdoors. To address the limitations, we propose a novel detection mechanism, DeDe, which detects the activation of backdoor mappings caused by triggered inputs on victim encoders. Specifically, DeDe trains a decoder for any given SSL encoder using an auxiliary dataset (which can be out-of-distribution or even slightly poisoned), so that for any triggered input that misleads the encoder into the target embedding, the decoder generates an output image significantly different from the input. DeDe leverages the discrepancy between the input and the decoded output to identify potential backdoor misbehavior during inference. We empirically evaluate DeDe on both contrastive learning and CLIP models against various types of backdoor attacks. Our results demonstrate promising detection effectiveness over various advanced attacks and superior performance compared over state-of-the-art detection methods.

DeDe: Detecting Backdoor Samples for SSL Encoders via Decoders

TL;DR

The paper addresses the vulnerability of self-supervised learning (SSL) encoders to backdoor attacks that map triggered inputs to target embeddings. It introduces DeDe, a decoder-based detector that learns an inverse mapping of the encoder by jointly training a patch encoder and a decoder on an auxiliary dataset, and signals backdoor activation via reconstruction errors, without requiring clean training data. DeDe shows robust detection across contrastive learning and CLIP settings against a range of stealthy and patch-like attacks, outperforming existing defenses such as DECREE and ASSET. The approach is non-invasive, data-agnostic, and effective at inference time, offering practical protection for downstream tasks relying on SSL encoders.

Abstract

Self-supervised learning (SSL) is pervasively exploited in training high-quality upstream encoders with a large amount of unlabeled data. However, it is found to be susceptible to backdoor attacks merely via polluting a small portion of training data. The victim encoders associate triggered inputs with target embeddings, e.g., mapping a triggered cat image to an airplane embedding, such that the downstream tasks inherit unintended behaviors when the trigger is activated. Emerging backdoor attacks have shown great threats across different SSL paradigms such as contrastive learning and CLIP, yet limited research is devoted to defending against such attacks, and existing defenses fall short in detecting advanced stealthy backdoors. To address the limitations, we propose a novel detection mechanism, DeDe, which detects the activation of backdoor mappings caused by triggered inputs on victim encoders. Specifically, DeDe trains a decoder for any given SSL encoder using an auxiliary dataset (which can be out-of-distribution or even slightly poisoned), so that for any triggered input that misleads the encoder into the target embedding, the decoder generates an output image significantly different from the input. DeDe leverages the discrepancy between the input and the decoded output to identify potential backdoor misbehavior during inference. We empirically evaluate DeDe on both contrastive learning and CLIP models against various types of backdoor attacks. Our results demonstrate promising detection effectiveness over various advanced attacks and superior performance compared over state-of-the-art detection methods.

Paper Structure

This paper contains 17 sections, 2 equations, 6 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Pre-train Encoders via SSL
  4. Backdoor attacks and defences in SSL
  5. Motivations
  6. What defines a good backdoor in encoders
  7. Limitations in Existing Backdoor Defences
  8. Design of DeDe
  9. Attack and defense models
  10. Key Idea
  11. Methodology
  12. Evaluation
  13. Experiment Set-up
  14. Experiment Results
  15. Conclusion
  16. ...and 2 more sections

Figures (6)

  • Figure 1: Embedding Visualizations.
  • Figure 2: The Workflow of DeDe.
  • Figure 3: The Learning Goal for Decoder of DeDe.
  • Figure 4: Discussion for CTRL Attack
  • Figure 5: Reconstruction of DeDe. The left side and the right side are attacked by DRUPE and CLIP-Backdoor respectively. In six columns, the left three are for clean inputs and the right three are for poisoned inputs. In three columns, they are masked image, reconstructed image, and ground truth (input) image accordingly.
  • ...and 1 more figures