Declarative Concurrent Data Structures

TL;DR

The paper tackles the difficulty of constructing scalable, correct concurrent data structures and the challenge of composing operations across multiple structures. It proposes Declarative Concurrent Data Structures (DCDS), a framework that uses a serial, declarative specification to automatically generate optimized, thread-safe data structures, with concurrency control injected at build time. Implemented in the Rösti system with an LLVM-based backend, DCDS enables co-optimization of both logical functionality and the CC protocol, yielding substantial performance gains in composed structures (e.g., map+list) and in-process OLTP workloads compared to state-of-the-art baselines. The work demonstrates that declarative design, coupled with workload-aware optimization, can deliver near hand-tuned performance while improving development ease and composability, and points to future directions in dynamic adaptation and storage-tiered deployments.

Abstract

Implementing concurrent data structures is challenging and requires a deep understanding of concurrency concepts and careful design to ensure correctness, performance, and scalability. Further, composing operations on two or more concurrent data structures often requires a synchronization wrapper to ensure the operations are applied together atomically, resulting in serialization and, thereby, giving up the performance benefit of the individual data structures. DBMS provides generalized concurrency control (CC) and is a good fit for implementing concurrent data structures. However, DBMSs are over-generalized for this use case, which fails to match the performance of specialized implementations. This paper makes the case for the Declarative Concurrent Data Structures (DCDS) framework for automatically generating concurrent data structures from a serial specification. In DCDS, users declare the attributes and methods needed for their desired data structure through an embedded DSL at design time. DCDS automatically injects CC at build-time, generating a concurrent intermediate representation (IR) compiled into machine code. A declarative interface for designing data structure enables efficient composability through co-optimizing component structures; optimizations are applied to both the composed serial specification and the generated concurrent IR. We realize the DCDS framework in our prototype system Rosti and experimentally show that data structures declared in Rosti can be efficiently composed by co-optimizing their logical functionality and the generated CC protocol. Our evaluation shows that composing a map and a list to create an LRU container can benefit up to 2X performance scalability in Rosti compared to an open-source library. We demonstrate the applicability of DCDS as an in-process OLTP by comparing it with in-memory DBMS, Proteus, and showing up to 2X performance gains.

Figures (7)

• Figure 1: The DCDS framework.
• Figure 2: Rösti System Architecture
• Figure 3: Usecase-driven specialization of doubly- to singly-linked list for FIFO usages
• Figure 4: Specializing doubly linked list to FIFO use case
• Figure 5: Composite Data Structure: LRU Container with a map and a doubly linked list