Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the Adversarial Vulnerabilities of Transfer Learning in Remote Sensing

Tao Bai, Xingjian Tian, Yonghao Xu, Bihan Wen

TL;DR

This work addresses the vulnerability of transfer learning in remote sensing by showing that publicly available pretrained models can be exploited to attack downstream tasks through Adversarial Neuron Manipulation (ANM). It introduces two attack variants, ANM-S (single-neuron) and ANM-M (multiple-neuron), under a grey-box threat model where perturbations are crafted from a pretrained model and transferred to a finetuned victim model without access to its data. A Mutual Information Maximization Search (MIMS) procedure selects influential neurons for ANM-M, achieving stronger and more consistent attacks than ANM-S across diverse datasets and architectures. The findings highlight a significant security risk in transfer learning for safety-critical remote sensing tasks and motivate defense strategies such as adversarial training, ensemble methods, and real-time anomaly detection to improve robustness.

Abstract

The use of pretrained models from general computer vision tasks is widespread in remote sensing, significantly reducing training costs and improving performance. However, this practice also introduces vulnerabilities to downstream tasks, where publicly available pretrained models can be used as a proxy to compromise downstream models. This paper presents a novel Adversarial Neuron Manipulation method, which generates transferable perturbations by selectively manipulating single or multiple neurons in pretrained models. Unlike existing attacks, this method eliminates the need for domain-specific information, making it more broadly applicable and efficient. By targeting multiple fragile neurons, the perturbations achieve superior attack performance, revealing critical vulnerabilities in deep learning models. Experiments on diverse models and remote sensing datasets validate the effectiveness of the proposed method. This low-access adversarial neuron manipulation technique highlights a significant security risk in transfer learning models, emphasizing the urgent need for more robust defenses in their design when addressing the safety-critical remote sensing tasks.

On the Adversarial Vulnerabilities of Transfer Learning in Remote Sensing

TL;DR

This work addresses the vulnerability of transfer learning in remote sensing by showing that publicly available pretrained models can be exploited to attack downstream tasks through Adversarial Neuron Manipulation (ANM). It introduces two attack variants, ANM-S (single-neuron) and ANM-M (multiple-neuron), under a grey-box threat model where perturbations are crafted from a pretrained model and transferred to a finetuned victim model without access to its data. A Mutual Information Maximization Search (MIMS) procedure selects influential neurons for ANM-M, achieving stronger and more consistent attacks than ANM-S across diverse datasets and architectures. The findings highlight a significant security risk in transfer learning for safety-critical remote sensing tasks and motivate defense strategies such as adversarial training, ensemble methods, and real-time anomaly detection to improve robustness.

Abstract

The use of pretrained models from general computer vision tasks is widespread in remote sensing, significantly reducing training costs and improving performance. However, this practice also introduces vulnerabilities to downstream tasks, where publicly available pretrained models can be used as a proxy to compromise downstream models. This paper presents a novel Adversarial Neuron Manipulation method, which generates transferable perturbations by selectively manipulating single or multiple neurons in pretrained models. Unlike existing attacks, this method eliminates the need for domain-specific information, making it more broadly applicable and efficient. By targeting multiple fragile neurons, the perturbations achieve superior attack performance, revealing critical vulnerabilities in deep learning models. Experiments on diverse models and remote sensing datasets validate the effectiveness of the proposed method. This low-access adversarial neuron manipulation technique highlights a significant security risk in transfer learning models, emphasizing the urgent need for more robust defenses in their design when addressing the safety-critical remote sensing tasks.
Paper Structure (18 sections, 10 equations, 10 figures, 3 tables, 2 algorithms)

This paper contains 18 sections, 10 equations, 10 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Transfer Learning
  4. Adversarial Attacks
  5. Methodologies
  6. Problem Formulation
  7. Adversarial Neuron Manipulation for Single Neuron (ANM-S)
  8. Adversarial Neuron Manipulation for Multiple Neurons (ANM-M)
  9. Experiments
  10. Experiment Settings
  11. Datasets
  12. Victim Models
  13. Implementation Details
  14. AMN-S
  15. ANM-M
  16. ...and 3 more sections

Figures (10)

  • Figure 1: The adversarial perturbation generated on publicly available pretrained deep models with Adversarial Neuron Manipulation(ANM) can threaten transfer-learned models effectively.
  • Figure 2: Overview of ANM-S. The ANM perturbation is optimized to maximize the value of a single neuron, which is randomly selected.
  • Figure 3: Overview of ANM-M. The ANM perturbation is optimized to maximize the value of a group of neurons, which have the maximal inter-neuron MI and are obtained via Mutual Information Maximization Search (MIMS).
  • Figure 4: Histograms of mean values and variances of ResNet50 feature extractor neurons.
  • Figure 5: Classification Accuracy of ResNet18 under ANM-M attack with different number of neurons.
  • ...and 5 more figures