Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Detecting APT Malware Command and Control over HTTP(S) Using Contextual Summaries

Almuthanna Alageel, Sergio Maffeis, Imperial College London

TL;DR

This work tackles the challenge of detecting APT malware C&C communications over HTTP(S), where traditional network detectors struggle due to low-and-slow TTPs and encrypted traffic. It introduces EarlyCrow, a context-aware detection framework built on PairFlow, a novel flow representation that aggregates per-flow and profile-based data into ContextualSummary features, enabling a robust random-forest classifier. The approach is evaluated on APT and botnet datasets, including unseen malware, and achieves macro F1-scores around 93% with very low FPR, both in HTTP and HTTPS modes, outperforming a baseline. The results demonstrate the value of context-rich features and structured flow representations for early, resilient detection, with practical implications for enterprise defenses and threat intelligence pipelines; future work includes expanding to additional protocols and exploring adversarial robustness.

Abstract

Advanced Persistent Threats (APTs) are among the most sophisticated threats facing critical organizations worldwide. APTs employ specific tactics, techniques, and procedures (TTPs) which make them difficult to detect in comparison to frequent and aggressive attacks. In fact, current network intrusion detection systems struggle to detect APTs communications, allowing such threats to persist unnoticed on victims' machines for months or even years. In this paper, we present EarlyCrow, an approach to detect APT malware command and control over HTTP(S) using contextual summaries. The design of EarlyCrow is informed by a novel threat model focused on TTPs present in traffic generated by tools recently used as part of APT campaigns. The threat model highlights the importance of the context around the malicious connections, and suggests traffic attributes which help APT detection. EarlyCrow defines a novel multipurpose network flow format called PairFlow, which is leveraged to build the contextual summary of a PCAP capture, representing key behavioral, statistical and protocol information relevant to APT TTPs. We evaluate the effectiveness of EarlyCrow on unseen APTs obtaining a headline macro average F1-score of 93.02% with FPR of $0.74%.

Detecting APT Malware Command and Control over HTTP(S) Using Contextual Summaries

TL;DR

This work tackles the challenge of detecting APT malware C&C communications over HTTP(S), where traditional network detectors struggle due to low-and-slow TTPs and encrypted traffic. It introduces EarlyCrow, a context-aware detection framework built on PairFlow, a novel flow representation that aggregates per-flow and profile-based data into ContextualSummary features, enabling a robust random-forest classifier. The approach is evaluated on APT and botnet datasets, including unseen malware, and achieves macro F1-scores around 93% with very low FPR, both in HTTP and HTTPS modes, outperforming a baseline. The results demonstrate the value of context-rich features and structured flow representations for early, resilient detection, with practical implications for enterprise defenses and threat intelligence pipelines; future work includes expanding to additional protocols and exploring adversarial robustness.

Abstract

Advanced Persistent Threats (APTs) are among the most sophisticated threats facing critical organizations worldwide. APTs employ specific tactics, techniques, and procedures (TTPs) which make them difficult to detect in comparison to frequent and aggressive attacks. In fact, current network intrusion detection systems struggle to detect APTs communications, allowing such threats to persist unnoticed on victims' machines for months or even years. In this paper, we present EarlyCrow, an approach to detect APT malware command and control over HTTP(S) using contextual summaries. The design of EarlyCrow is informed by a novel threat model focused on TTPs present in traffic generated by tools recently used as part of APT campaigns. The threat model highlights the importance of the context around the malicious connections, and suggests traffic attributes which help APT detection. EarlyCrow defines a novel multipurpose network flow format called PairFlow, which is leveraged to build the contextual summary of a PCAP capture, representing key behavioral, statistical and protocol information relevant to APT TTPs. We evaluate the effectiveness of EarlyCrow on unseen APTs obtaining a headline macro average F1-score of 93.02% with FPR of $0.74%.

Paper Structure

This paper contains 55 sections, 2 equations, 9 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Threat Model
  3. Current Data Formats Issues
  4. TTP Relevant Data
  5. IoC-like Data
  6. Traffic Data
  7. Traffic Measurements
  8. Traffic Statistical Measurements
  9. Time-based Measurements
  10. Remote Web Server
  11. PairFlow
  12. Architecutre Overview
  13. Tracking
  14. Packets Retrieving.
  15. DNS Requests and Responses.
  16. ...and 40 more sections

Figures (9)

  • Figure 1: Types of APT malware initial communications. Fallback channels may use one or more TTPs (blue text). Malicious data packets are represented by red arrows.
  • Figure 2: Measurements for APT, botnets, and legitimate connections.
  • Figure 3: Overview of the PairFlow workflow.
  • Figure 4: Overview of the EarlyCrow architecture.
  • Figure 5: Causal reasoning relationships castro2020causality.
  • ...and 4 more figures