Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LLMs for Domain Generation Algorithm Detection

Reynier Leyva La O, Carlos A. Catania, Tatiana Parlanti

Abstract

This work analyzes the use of large language models (LLMs) for detecting domain generation algorithms (DGAs). We perform a detailed evaluation of two important techniques: In-Context Learning (ICL) and Supervised Fine-Tuning (SFT), showing how they can improve detection. SFT increases performance by using domain-specific data, whereas ICL helps the detection model to quickly adapt to new threats without requiring much retraining. We use Meta's Llama3 8B model, on a custom dataset with 68 malware families and normal domains, covering several hard-to-detect schemes, including recent word-based DGAs. Results proved that LLM-based methods can achieve competitive results in DGA detection. In particular, the SFT-based LLM DGA detector outperforms state-of-the-art models using attention layers, achieving 94% accuracy with a 4% false positive rate (FPR) and excelling at detecting word-based DGA domains.

LLMs for Domain Generation Algorithm Detection

Abstract

This work analyzes the use of large language models (LLMs) for detecting domain generation algorithms (DGAs). We perform a detailed evaluation of two important techniques: In-Context Learning (ICL) and Supervised Fine-Tuning (SFT), showing how they can improve detection. SFT increases performance by using domain-specific data, whereas ICL helps the detection model to quickly adapt to new threats without requiring much retraining. We use Meta's Llama3 8B model, on a custom dataset with 68 malware families and normal domains, covering several hard-to-detect schemes, including recent word-based DGAs. Results proved that LLM-based methods can achieve competitive results in DGA detection. In particular, the SFT-based LLM DGA detector outperforms state-of-the-art models using attention layers, achieving 94% accuracy with a 4% false positive rate (FPR) and excelling at detecting word-based DGA domains.

Paper Structure

This paper contains 17 sections, 12 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Large Language Models
  4. Model
  5. In-Context Learning
  6. Supervised Fine-Tuning
  7. Previous Works
  8. Methodology
  9. Dataset Description
  10. Training Process
  11. Evaluation Process
  12. Experimental Results
  13. Experiment I: Evaluation of Training Approaches
  14. Experiment II: Evaluation of Generalization to New Families
  15. Experiment III: Comparison with Previous Approaches
  16. ...and 2 more sections

Figures (12)

  • Figure 1: Distribution of domains for the different training methods.
  • Figure 2: Example format for training data (for SFT), including domain and label.
  • Figure 3: Prompt used on Llama3 8B for classification of domain names in ICL.
  • Figure 4: Model evaluation diagram.
  • Figure 5: Systematic sampling: 30 samples of 50 legit and 50 DGA domains. Each circle represents 50 different domains, and all circles are disjoint.
  • ...and 7 more figures