Measuring Round-Trip Response Latencies Under Asymmetric Routing

TL;DR

The paper addresses measuring client-visible response latency in environments with encrypted transports and routing asymmetry. It introduces PIRATE, a passive approach that infers application-layer latency from causal pairs—two requests where the second is triggered by the first's response—by analyzing the inter-packet gap distribution to identify events and compute an average latency $L$ over time. In experiments with realistic web workloads and load-balancing deployments, PIRATE achieves median relative accuracy below $1\%$ and, when integrated into a latency-aware layer-4 load balancer, reduces the 99th percentile tail latency by about $37\%$, with low per-packet overhead ($\sim$ few hundred nanoseconds) and modest memory. The method generalizes across transports (TCP/QUIC) and apps, remains robust under loss/reordering, and supports online deployment on software middleboxes, making it practical for continuous, passive latency monitoring and reactive control.

Abstract

Latency is a key indicator of Internet service performance. Continuously tracking the latency of client requests enables service operators to quickly identify bottlenecks, perform adaptive resource allocation or routing, and mitigate attacks. Passively measuring the response latency at intermediate vantage points is attractive since it provides insight into the experience of real clients without requiring client instrumentation or incurring probing overheads. This paper presents PIRATE, a passive approach to measure response latencies when only the client-to-server traffic is visible, even when transport headers are encrypted. PIRATE estimates the time gap between causal pairs - two requests such that the response to the first triggered the second - as a proxy for the client-side response latency. Our experiments with a realistic web application show that PIRATE can estimate the response latencies measured at the client application layer to within 1 percent. A PIRATE-enhanced layer-4 load balancer (with DSR) cuts tail latencies by 37 percent.

Figures (11)

• Figure 1: TCP Round-Trip Time (RTT) estimation from the network stack (tcp-probe), trace-based methods (tcpTrace), and syn-ack estimation passive-estimation-tcp-rtt-ccr02 do not match the client-visible response latency.
• Figure 2: Modes in the empirical distribution of inter-packet gaps (IPGs). Modes carry useful information about phenomena of interest occurring over the measurement epoch.
• Figure 3: (a) CDF of observed latencies across all the connections for the web workload in §\ref{['sec:detailed-experimental-setup']}. (b) CDF of absolute errors across all responses (relative to response latency). (c) CDF of relative errors across all responses (relative to response latency).
• Figure 4: (a) CDF of samples that differ from request-to-request delay due to parsing/scheduling delays at client or processing delays at server (b) CDF of number of sample within each connection that differ from request-to-request delay due to parsing/scheduling delays at client or processing delays at server
• Figure 5: (a) CDF of all measurements throughout the lifetime of all connections. (b) and (c): Time evolution of the latency of a single (randomly chosen) connection during a small window of data transfer.