Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Octopus: A Fair Packet Delivery Service

Junzhi Gong, Yuliang Li, Devdeep Ray, KK Yap, Nandita Dukkipati

TL;DR

Octopus tackles the problem of fair multicast delivery in shared cloud environments by defining fairness as $\max_i RT_i - \min_i RT_i$ and releasing packets at a global time $T_{release} = T_{now} + \delta$. Its approach combines NIC hardware time-based release with agents deployed on local SmartNICs for each receiver, significantly reducing software jitter and post-release queuing. The key contributions are (i) repurposing NIC traffic shaping for nanosecond-scale release accuracy and (ii) placing agents on SmartNICs to minimize path variations, achieving less than ~$40$ ns unfairness for the vast majority of packets in evaluation. The results suggest practical impact for latency-sensitive cloud applications, enabling broad fair multicast support without heavy on-premise customization or application changes.

Abstract

The packet delivery fairness is critical in many applications in the cloud, such as exchange systems, consensus protocols, and online gaming applications. However, due to nonidentical and dynamic packet forwarding paths, as well as many in-network queuing delays, supporting packet delivery fairness is challenging in a shared compute environment. In this paper, we present Octopus, the first general fair packet delivery service to achieve packet arrival time variations smaller than tens of nanoseconds, with the existence of latency variations in the network. The key ideas of Octopus to support such good fairness come from repurposing hardware traffic shaping capabilities in modern NICs, and deploying agents at local SmartNICs to minimize latency variations from packet forwarding. Evaluation results show that Octopus has less than 40 ns unfairness for up to 99.97\% multicast packets.

Octopus: A Fair Packet Delivery Service

TL;DR

Octopus tackles the problem of fair multicast delivery in shared cloud environments by defining fairness as and releasing packets at a global time . Its approach combines NIC hardware time-based release with agents deployed on local SmartNICs for each receiver, significantly reducing software jitter and post-release queuing. The key contributions are (i) repurposing NIC traffic shaping for nanosecond-scale release accuracy and (ii) placing agents on SmartNICs to minimize path variations, achieving less than ~ ns unfairness for the vast majority of packets in evaluation. The results suggest practical impact for latency-sensitive cloud applications, enabling broad fair multicast support without heavy on-premise customization or application changes.

Abstract

The packet delivery fairness is critical in many applications in the cloud, such as exchange systems, consensus protocols, and online gaming applications. However, due to nonidentical and dynamic packet forwarding paths, as well as many in-network queuing delays, supporting packet delivery fairness is challenging in a shared compute environment. In this paper, we present Octopus, the first general fair packet delivery service to achieve packet arrival time variations smaller than tens of nanoseconds, with the existence of latency variations in the network. The key ideas of Octopus to support such good fairness come from repurposing hardware traffic shaping capabilities in modern NICs, and deploying agents at local SmartNICs to minimize latency variations from packet forwarding. Evaluation results show that Octopus has less than 40 ns unfairness for up to 99.97\% multicast packets.
Paper Structure (14 sections, 1 equation, 5 figures, 1 table)

This paper contains 14 sections, 1 equation, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. Problem Definition and Design Goals
  3. Octopus Design
  4. System Architecture
  5. Key Ideas
  6. Accurate and Fair Packet Releasing with NIC Hardware Offloading
  7. Minimize Unfairness after Packet Releasing
  8. Summary and Limitations
  9. Evaluation
  10. Evaluation Setup
  11. End-to-end Results
  12. The Impact from $\delta$
  13. Related Works
  14. Conclusion

Figures (5)

  • Figure 1: Octopus' architecture. Note that grey boxes (the sender application and receiver applications) are not part of the Octopus' architecture. Boxes marked with a clock are clock-synchronized.
  • Figure 2: End to end evaluation results. The x-axis represents different throughputs to multicast packets at the sender, and the y-axis represents the unfairness rate ($>$40 ns and $>$100 ns) among receivers.
  • Figure 3: CDF of unfairness under different throughputs, with $\delta=100\mu$s.
  • Figure 4: CDF of unfairness under 500 Kpps throughput.
  • Figure 5: CDF of unfairness under 200 Kpps throughput.