Milliscale: Fast Commit on Low-Latency Object Storage

TL;DR

Evaluation of Milliscale, a memory-optimized OLTP engine for low-latency object storage, shows that it delivers much lower commit latency than baselines while sustaining high throughput.

Abstract

With millisecond-level latency and support for mutable objects, recent low-latency object storage services as represented by Amazon S3 Express One Zone have become an attractive option for OLTP engines to directly commit transactions and persist operational data with transparent strong consistency, high durability and high availability. But a naïve adoption can still lead to high commit latency due to idiosyncrasies of S3 Express One Zone and modern decentralized logging. This paper presents Milliscale, a memory-optimized OLTP engine for low-latency object storage. Milliscale optimizes commit latency with new techniques that lower commit delays and reduce the number of object access requests. Our evaluation using representative benchmarks shows that Milliscale delivers much lower commit latency than baselines while sustaining high throughput.

Figures (18)

• Figure 1: Commit latency vs. cost. Traditional object (S3 Standard) and block (local SSD, io2 and gp3) storage incur high commit latency and cost, respectively.
• Figure 2: Object get, append, and put latency (ms) of S3 Standard vs. S3 Express One Zone under varying request sizes. S3 Express One Zone can offer single-digit ms latency at small request sizes (e.g., 512KB), making it a good fit for OLTP.
• Figure 3: Milliscale Overview. Worker threads access records via indexes and version chains. The log is segmented into objects which are sealed once reaching append limitation. To reduce S3 requests while retaining high throughput and low commit latency, threads are grouped to share log buffers. Pre-committed transactions stay in a commit queue until their own and depending log records are persisted.
• Figure 4: Throughput (left) and average commit latency (right) of TPC-C under different log buffer sizes and sharing ratios. S3 latency dominate with a 4:1 ratio, whereas a 2:1 ratio leads to the highest throughput and lowest latency.
• Figure 5: Transaction (a) vs. record (b) level dependency tracking which. Tracking at the record-level allows transaction with CSN 103 to commit, by skipping waiting for the transaction with CSN 102 to commit first.