POMDP-Based Routing for DTNs with Partial Knowledge and Dependent Failures

TL;DR

This work tackles DTN routing under partial knowledge and correlated failures by formulating a POMDP-based routing framework that embeds Dependent Node Failures (DNF) through repairable-system CTMC dynamics. Implemented in JuliaPOMDP and integrated with DtnSim, the approach defines explicit state, action, and observation spaces, a failure-aware transition model, and a time-aware reward structure, augmented by an offline lttg reachability heuristic and an online PO-UCT planner (BasicPOMCP). Empirical results show that the DNF-based routing can improve delivery ratios and energy efficiency under uncertainty and remains computationally feasible for onboard execution, though it may be outperformed by specialized baselines in some metrics depending on failure regimes. The work demonstrates a principled decision-theoretic foundation for routing in future DTN deployments and identifies concrete avenues for reducing computation, extending observation sharing, and benchmarking across diverse networks.

Abstract

Routing in Delay-Tolerant Networks (DTNs) is inherently challenging due to sparse connectivity, long delays, and frequent disruptions. While Markov Decision Processes (MDPs) have been used to model uncertainty, they assume full state observability - an assumption that breaks down in partitioned DTNs, where each node operates with inherently partial knowledge of the network state. In this work, we investigate the role of Partially Observable Markov Decision Processes (POMDPs) for DTN routing under uncertainty. We introduce and evaluate a novel model: Dependent Node Failures (DNF), which captures correlated node failures via repairable node states modeled as Continuous-Time Markov Chains (CTMCs). We implement the model using JuliaPOMDP and integrate it with DTN simulations via DtnSim. Our evaluation demonstrates that POMDP-based routing yields improved delivery ratios and delay performance under uncertain conditions while maintaining scalability. These results highlight the potential of POMDPs as a principled foundation for decision-making in future DTN deployments.

Figures (1)

• Figure 1: Heatmaps comparing the metrics for dnf, cgr, and cgr-CR under varying mtbf & mttr and a transmission failure probability of $5\%$.