Delegation with Trust<T>: A Scalable, Type- and Memory-Safe Alternative to Locks

TL;DR

Evaluation shows Trust consistently and considerably outperforming locking where lock contention exists, with up to 22x higher throughput in microbenchmarks, and 5-9x for a home grown key-value store, as well as memcached, in situations with high lock contention.

Abstract

We present Trust<T>, a general, type- and memory-safe alternative to locking in concurrent programs. Instead of synchronizing multi-threaded access to an object of type T with a lock, the programmer may place the object in a Trust<T>. The object is then no longer directly accessible. Instead a designated thread, the object's trustee, is responsible for applying any requested operations to the object, as requested via the Trust<T> API. Locking is often said to offer a limited throughput per lock. Trust<T> is based on delegation, a message-passing technique which does not suffer this per-lock limitation. Instead, per-object throughput is limited by the capacity of the object's trustee, which is typically considerably higher. Our evaluation shows Trust<T> consistently and considerably outperforming locking where lock contention exists, with up to 22x higher throughput in microbenchmarks, and 5-9x for a home grown key-value store, as well as memcached, in situations with high lock contention. Moreover, Trust<T> is competitive with locks even in the absence of lock contention.

Figures (11)

• Figure 1: Minimal Trust$<$T$>$ example. An entrusted counter, referenced by ct is initialized to 17, then incremented once. The comments on the right indicate the types of the variables.
• Figure 2: Minimal multi-threaded Trust$<$T$>$ example. Reference counting ensures that the property remains in memory until the last Trust$<$T$>$ referencing the property drops.
• Figure 3: Asynchronous version of the example in Fig. \ref{['f:minimal']}. The second closure runs on the client, once the result of the first closure is received from the trustee.
• Figure 4: Operation of launch() vs apply(). launch() supports blocking calls, including nested delegation calls in the delegated closure, but incurs a higher minimum overhead. Solid arrows indicate requests, dotted arrows are delegation responses.
• Figure 5: The fixed-size Trust$<$T$>$ request slot consists of a ready bit, a request counter, and a variable number of variable-sized requests. The response slot contains a matching bit, as well as one (fixed, variable, or zero-sized) response per request in the matching request slot. There is one dedicated pair of request/response slots for each trustee/client pair.