Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exploring Adversarial Robustness in Classification tasks using DNA Language Models

Hyunwoo Yoo, Haebin Shin, Kaidi Xu, Gail Rosen

TL;DR

DNA language models for sequence classification face robustness challenges under real-world sequencing noise. The authors apply multi-granularity adversarial perturbations—nucleotide-level with magnitude $\epsilon$, codon-level, and backtranslation—to three models (DNABERT2, GROVER, Nucleotide Transformer) and evaluate on AMR drug classification and promoter detection, demonstrating substantial degradation especially from nucleotide-level attacks. They additionally assess adversarial training as a defense, finding it can improve robustness and, in some cases, overall accuracy, with effectiveness dependent on attack type and dataset. The work highlights the need to integrate robustness considerations into bioinformatics workflows to ensure reliable genomic analyses in practical settings.

Abstract

DNA Language Models, such as GROVER, DNABERT2 and the Nucleotide Transformer, operate on DNA sequences that inherently contain sequencing errors, mutations, and laboratory-induced noise, which may significantly impact model performance. Despite the importance of this issue, the robustness of DNA language models remains largely underexplored. In this paper, we comprehensivly investigate their robustness in DNA classification by applying various adversarial attack strategies: the character (nucleotide substitutions), word (codon modifications), and sentence levels (back-translation-based transformations) to systematically analyze model vulnerabilities. Our results demonstrate that DNA language models are highly susceptible to adversarial attacks, leading to significant performance degradation. Furthermore, we explore adversarial training method as a defense mechanism, which enhances both robustness and classification accuracy. This study highlights the limitations of DNA language models and underscores the necessity of robustness in bioinformatics.

Exploring Adversarial Robustness in Classification tasks using DNA Language Models

TL;DR

DNA language models for sequence classification face robustness challenges under real-world sequencing noise. The authors apply multi-granularity adversarial perturbations—nucleotide-level with magnitude , codon-level, and backtranslation—to three models (DNABERT2, GROVER, Nucleotide Transformer) and evaluate on AMR drug classification and promoter detection, demonstrating substantial degradation especially from nucleotide-level attacks. They additionally assess adversarial training as a defense, finding it can improve robustness and, in some cases, overall accuracy, with effectiveness dependent on attack type and dataset. The work highlights the need to integrate robustness considerations into bioinformatics workflows to ensure reliable genomic analyses in practical settings.

Abstract

DNA Language Models, such as GROVER, DNABERT2 and the Nucleotide Transformer, operate on DNA sequences that inherently contain sequencing errors, mutations, and laboratory-induced noise, which may significantly impact model performance. Despite the importance of this issue, the robustness of DNA language models remains largely underexplored. In this paper, we comprehensivly investigate their robustness in DNA classification by applying various adversarial attack strategies: the character (nucleotide substitutions), word (codon modifications), and sentence levels (back-translation-based transformations) to systematically analyze model vulnerabilities. Our results demonstrate that DNA language models are highly susceptible to adversarial attacks, leading to significant performance degradation. Furthermore, we explore adversarial training method as a defense mechanism, which enhances both robustness and classification accuracy. This study highlights the limitations of DNA language models and underscores the necessity of robustness in bioinformatics.
Paper Structure (25 sections, 1 equation, 9 figures, 6 tables)

This paper contains 25 sections, 1 equation, 9 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Adversarial Attacks in Text Classification
  4. Adversarial Attacks in DNA Sequence Classification
  5. Adversarial Training
  6. Adversarial Attack on DNA Language Models
  7. Nucleotide-Level Attack
  8. Codon-Level Attack
  9. Backtranslation Attack
  10. Results
  11. Attack Effectiveness and Context Preservation
  12. Model Robustness Comparison
  13. Comparative Analysis of Classification Tasks
  14. Performance Against Defense Method
  15. Conclusion
  16. ...and 10 more sections

Figures (9)

  • Figure 1: Adversarial examples with nucleotides and codon change, which will be misclassified by a non-promoter to a promoter.
  • Figure 2: Anti-Microbial Resistance (AMR) gene Classification and Promoter Detection Results
  • Figure 3: Comparison of success rates and accuracies between adversarial training and standard training in AMR gene Classification across increasing iterations of backtranslation attacks
  • Figure 4: Overview of Adversarial Training(Fine-tuning) for DNA Language Models and Testing Adversarial Attacks
  • Figure 5: Comparison of Adversarial Training and Standard Training in Promoter Detection with Increasing Iterations
  • ...and 4 more figures