DTN-COMET: A Comprehensive Operational Metrics Evaluation Toolkit for DTN

TL;DR

This work addresses the need for reproducible, scalable evaluation of Delay-Tolerant Networking (DTN) implementations by introducing a generic automated performance evaluation framework and a physical testbed. The framework measures metrics such as goodput ($Mbit/s$), round-trip time ($RTT$), bundle retention time ($BRT$), and (de-)serialization time ($T_{ser}$, $T_{des}$), and validates its utility through a comparative study of the µD3TN and ION DTN implementations under a common TCP convergence layer. The results show µD3TN achieving higher goodput and shorter BRT, while ION demonstrates stronger memory management and fault tolerance, illustrating the trade-offs between lightweight RAM-based processing and structured memory usage. Overall, the framework enables reproducible, implementation-agnostic evaluation and lays groundwork for extending toward link-disruption modeling and interoperability testing, addressing a key gap in DTN toolkit standardization.

Abstract

Delay- and Disruption-tolerant Networking (DTN) is essential for communication in challenging environments with intermittent connectivity, long delays, and disruptions. Ensuring high performance in these types of networks is crucial because windows for data transmission are sparse and often short. However, research on evaluating DTN implementations is limited. Moreover, existing research relies on manual testing methods that lack reproducibility and scalability. We propose a novel generic framework for reproducible performance evaluation of DTN implementations to address this issue. We validate the framework's accuracy using a physical testbed and compare the uD3TN and ION DTN implementations. This comparison reveals that uD3TN exhibits higher goodput and shorter bundle retention times. On the other hand, ION exhibited superior memory management and fault tolerance, albeit at the cost of sending and receiving performance. Through this comparison, our framework demonstrates the feasibility of developing a generic toolkit for evaluating DTN.

Figures (5)

• Figure 1: An overview of the modules and their interdependence on the testbed nodes.
• Figure 2: Cumulative probabilities of round-trip times per implementation for 1000 bundles.
• Figure 3: Average goodput for different bundle payload sizes including 99% confidence intervals. For each payload size 10,000 bundles were sent.
• Figure 4: Measured memory usage for different payload sizes. Note the fluctuations in recorded memory usage for ION and µD3TN due to the inclusion of page cache in the measurements.
• Figure 5: Average bundle retention, serialization, and deserialization times measured for ION and µD3TN. For each payload size 1,000 bundles were sent.