Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Design and Implementation of a Scalable Financial Exchange in the Public Cloud

Muhammad Haseeb, Jinkun Geng, Daniel Duclos-Cavalcanti, Ulysses Butler, Xiyu Hao, Radhika Mittal, Srinivas Narayana, Anirudh Sivaraman

TL;DR

Jasper addresses the challenge of delivering low-latency, fair market data and order submissions in public clouds by combining an overlay multicast tree for outbound data, a clock-synchronised inbound sequencer, and a novel Limit Order Queue to handle bursts. The approach leverages three outbound hedges (Round-Robin Packet Spraying, Proxy Hedging, Receiver Hedging), a tree-reuse strategy for inbound traffic, and hold-and-release deadlines to enforce near-simultaneous delivery while maintaining inbound fairness. Empirical results show median multicast latency around $250\mu s$ with ≤$1\mu s$ spread for ~1000 receivers, throughput up to $175K$ MPS, and substantial improvements in order matching rate during bursts compared with CloudEx and AWS TGW. The work demonstrates that cloud-hosted financial exchanges can achieve strong fairness and performance with a composable networking layer, and provides Onyx as an open-source platform to facilitate adoption and further research.

Abstract

Financial exchanges are migrating to the cloud, but the best-effort nature of the public cloud is at odds with the stringent latency requirements of exchanges. We present Jasper, a system for meeting the networking requirements of financial exchanges on the public cloud. Jasper uses an overlay tree to scalably multicast market data from an exchange to ~1000 participants with low latency (250 microseconds) and a 1-microsecond difference in data reception time between any two participants. Jasper reuses the same tree for scalable inbound communication (participants to exchange), augmenting it with order pacing and a new priority queue, Limit Order Queue (LOQ), to efficiently handle bursts of market orders. Jasper achieves better scalability and 50% lower latency than the AWS multicast service. During bursty market activity, LOQ nearly doubles the order processing rate.

Design and Implementation of a Scalable Financial Exchange in the Public Cloud

TL;DR

Jasper addresses the challenge of delivering low-latency, fair market data and order submissions in public clouds by combining an overlay multicast tree for outbound data, a clock-synchronised inbound sequencer, and a novel Limit Order Queue to handle bursts. The approach leverages three outbound hedges (Round-Robin Packet Spraying, Proxy Hedging, Receiver Hedging), a tree-reuse strategy for inbound traffic, and hold-and-release deadlines to enforce near-simultaneous delivery while maintaining inbound fairness. Empirical results show median multicast latency around with ≤ spread for ~1000 receivers, throughput up to MPS, and substantial improvements in order matching rate during bursts compared with CloudEx and AWS TGW. The work demonstrates that cloud-hosted financial exchanges can achieve strong fairness and performance with a composable networking layer, and provides Onyx as an open-source platform to facilitate adoption and further research.

Abstract

Financial exchanges are migrating to the cloud, but the best-effort nature of the public cloud is at odds with the stringent latency requirements of exchanges. We present Jasper, a system for meeting the networking requirements of financial exchanges on the public cloud. Jasper uses an overlay tree to scalably multicast market data from an exchange to ~1000 participants with low latency (250 microseconds) and a 1-microsecond difference in data reception time between any two participants. Jasper reuses the same tree for scalable inbound communication (participants to exchange), augmenting it with order pacing and a new priority queue, Limit Order Queue (LOQ), to efficiently handle bursts of market orders. Jasper achieves better scalability and 50% lower latency than the AWS multicast service. During bursty market activity, LOQ nearly doubles the order processing rate.
Paper Structure (37 sections, 2 equations, 18 figures, 5 tables, 1 algorithm)

This paper contains 37 sections, 2 equations, 18 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background
  3. Jasper Overview
  4. Market Data Multicast
  5. Round-Robin Packet Spraying
  6. Proxy Hedging
  7. Receiver Hedging
  8. Remarks on Multicast Packet Losses
  9. Orders Submission Service
  10. Sequencer
  11. Sequencer Protocol
  12. Implementation
  13. Limit Order Queue
  14. LOQ protocol
  15. Inbound fairness under LOQ
  16. ...and 22 more sections

Figures (18)

  • Figure 1: Latency between a pair of VMs varies over time
  • Figure 1: Jasper achieves fairness for a large $N$ with low OML($µs$).
  • Figure 2: Overview of Jasper.
  • Figure 3: Direct unicasts do not scale well; a tree does.
  • Figure 4: A Limit Order Book snapshot.
  • ...and 13 more figures

Theorems & Definitions (2)

  • definition 1: Outbound Fairness
  • definition 2: Inbound Fairness