Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Merit Network Telescope: Processing and Initial Insights from Nearly 20 Years of Darknet Traffic for Cybersecurity Research

Shereen Ismail, Eman Hammad, William Hatcher, Salah Dandan, Ammar Alomari, Michael Spratt

TL;DR

This work addresses the challenge of extracting meaningful, long-term insights from nearly two decades of unsolicited Internet traffic captured by a large, persistent network telescope. It introduces a coarse-to-fine processing framework with high-level metadata and packet-header pipelines and a dual database visualization strategy using InfluxDB and MariaDB and Grafana dashboards. Key findings include sustained scanning and backscatter activity, notable 2024 traffic characteristics such as Telnet port dominance, and documented data outages plus the effect of IP space reduction on trend interpretation. The approach provides a scalable foundation for future large-scale network measurement and threat intelligence and enables collaborative exploration of decades of darknet traffic.

Abstract

This paper presents an initial longitudinal analysis of unsolicited Internet traffic collected between 2005 and 2025 by one of the largest and most persistent network telescopes in the United States, operated by Merit Network. The dataset provides a unique view into global threat activity as observed through scanning and backscatter traffic, key indicators of large-scale probing behavior, data outages, and ongoing denial-of-service (DoS) campaigns. To process this extensive archive, coarse-to-fine methodology is adopted in which general insights are first extracted through a resource-efficient metadata sub-pipeline, followed by a more detailed packet header sub-pipeline for finer-grained analysis. The methodology establishes two sub-pipelines to enable scalable processing of nearly two decades of telescope data and supports multi-level exploration of traffic dynamics. Initial insights highlight long-term trends and recurring traffic spikes, some attributable to Internet-wide scanning events and others likely linked to DoS activities.We present general observations spanning 2006-2024, with a focused analysis of traffic characteristics during 2024.

Merit Network Telescope: Processing and Initial Insights from Nearly 20 Years of Darknet Traffic for Cybersecurity Research

TL;DR

This work addresses the challenge of extracting meaningful, long-term insights from nearly two decades of unsolicited Internet traffic captured by a large, persistent network telescope. It introduces a coarse-to-fine processing framework with high-level metadata and packet-header pipelines and a dual database visualization strategy using InfluxDB and MariaDB and Grafana dashboards. Key findings include sustained scanning and backscatter activity, notable 2024 traffic characteristics such as Telnet port dominance, and documented data outages plus the effect of IP space reduction on trend interpretation. The approach provides a scalable foundation for future large-scale network measurement and threat intelligence and enables collaborative exploration of decades of darknet traffic.

Abstract

This paper presents an initial longitudinal analysis of unsolicited Internet traffic collected between 2005 and 2025 by one of the largest and most persistent network telescopes in the United States, operated by Merit Network. The dataset provides a unique view into global threat activity as observed through scanning and backscatter traffic, key indicators of large-scale probing behavior, data outages, and ongoing denial-of-service (DoS) campaigns. To process this extensive archive, coarse-to-fine methodology is adopted in which general insights are first extracted through a resource-efficient metadata sub-pipeline, followed by a more detailed packet header sub-pipeline for finer-grained analysis. The methodology establishes two sub-pipelines to enable scalable processing of nearly two decades of telescope data and supports multi-level exploration of traffic dynamics. Initial insights highlight long-term trends and recurring traffic spikes, some attributable to Internet-wide scanning events and others likely linked to DoS activities.We present general observations spanning 2006-2024, with a focused analysis of traffic characteristics during 2024.

Paper Structure

This paper contains 14 sections, 9 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Coarse-to-Fine Sub-pipelines
  5. High-Level Metadata Sub-pipeline
  6. Packet Header Sub-pipeline
  7. Database Storage & Visualization
  8. Key Observations and Discussion
  9. General High-Level Metadata Insights:
  10. Packet-header Insights (year 2024):
  11. Data Outages:
  12. Data Processing Constraints in Coarse-to-Fine Sub-Pipelines
  13. Impact of IP Space Reduction
  14. Conclusion and Future Work

Figures (9)

  • Figure 1: Coarse-to-Fine Sub-pipelines of the ORION Network Telescope
  • Figure 2: Apache Drill SQL Query
  • Figure 3: MariaDB CSV Import SQL Query
  • Figure 4: Longitudinal trends- and traffic spikes in ORION darknet data over the years 2006 to 2024.
  • Figure 5: Traffic trends and spikes in ORION darknet data in the year 2024.
  • ...and 4 more figures