Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FaaSKeeper: Learning from Building Serverless Services with ZooKeeper as an Example

Marcin Copik, Alexandru Calotoiu, Pengyu Zhou, Konstantin Taranov, Torsten Hoefler

TL;DR

FaaSKeeper demonstrates that a complex, strongly consistent coordination service like ZooKeeper can be rebuilt on serverless platforms by decoupling compute from storage, using cloud storage and queues to enforce ordering and atomicity. The design maps ZooKeeper’s follower/leader roles to stateless functions, with system and user data stored in replicated cloud storage and watches delivered through epoch counters, achieving similar semantics under a pay‑as‑you‑go model. Evaluations show substantial cost benefits, particularly for infrequent workloads, and highlight both the promise and current limitations of serverless platforms for stateful, latency‑sensitive coordination. The work provides a concrete path and nine cloud requirements to guide future serverless designs of distributed, stateful services with broad practical impact.

Abstract

FaaS (Function-as-a-Service) revolutionized cloud computing by replacing persistent virtual machines with dynamically allocated resources. This shift trades locality and statefulness for a pay-as-you-go model more suited to variable and infrequent workloads. However, the main challenge is to adapt services to the serverless paradigm while meeting functional, performance, and consistency requirements. In this work, we push the boundaries of FaaS computing by designing a serverless variant of ZooKeeper, a centralized coordination service with a safe and wait-free consensus mechanism. We define synchronization primitives to extend the capabilities of scalable cloud storage and outline a set of requirements for efficient computing with serverless. In FaaSKeeper, the first coordination service built on serverless functions and cloud-native services, we explore the limitations of serverless offerings and propose improvements essential for complex and latency-sensitive applications. We share serverless design lessons based on our experiences of implementing a ZooKeeper model deployable to clouds today. FaaSKeeper maintains the same consistency guarantees and interface as ZooKeeper, with a serverless price model that lowers costs up to 110-719x on infrequent workloads.

FaaSKeeper: Learning from Building Serverless Services with ZooKeeper as an Example

TL;DR

FaaSKeeper demonstrates that a complex, strongly consistent coordination service like ZooKeeper can be rebuilt on serverless platforms by decoupling compute from storage, using cloud storage and queues to enforce ordering and atomicity. The design maps ZooKeeper’s follower/leader roles to stateless functions, with system and user data stored in replicated cloud storage and watches delivered through epoch counters, achieving similar semantics under a pay‑as‑you‑go model. Evaluations show substantial cost benefits, particularly for infrequent workloads, and highlight both the promise and current limitations of serverless platforms for stateful, latency‑sensitive coordination. The work provides a concrete path and nine cloud requirements to guide future serverless designs of distributed, stateful services with broad practical impact.

Abstract

FaaS (Function-as-a-Service) revolutionized cloud computing by replacing persistent virtual machines with dynamically allocated resources. This shift trades locality and statefulness for a pay-as-you-go model more suited to variable and infrequent workloads. However, the main challenge is to adapt services to the serverless paradigm while meeting functional, performance, and consistency requirements. In this work, we push the boundaries of FaaS computing by designing a serverless variant of ZooKeeper, a centralized coordination service with a safe and wait-free consensus mechanism. We define synchronization primitives to extend the capabilities of scalable cloud storage and outline a set of requirements for efficient computing with serverless. In FaaSKeeper, the first coordination service built on serverless functions and cloud-native services, we explore the limitations of serverless offerings and propose improvements essential for complex and latency-sensitive applications. We share serverless design lessons based on our experiences of implementing a ZooKeeper model deployable to clouds today. FaaSKeeper maintains the same consistency guarantees and interface as ZooKeeper, with a serverless price model that lowers costs up to 110-719x on infrequent workloads.
Paper Structure (43 sections, 2 equations, 16 figures, 4 tables, 2 algorithms)

This paper contains 43 sections, 2 equations, 16 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background and Motivation
  3. Serverless Components
  4. ZooKeeper
  5. From ZooKeeper to FaasKeeper
  6. Follower
  7. Leader
  8. Storage
  9. Watch Notifications
  10. Client
  11. Heartbeat
  12. Summary
  13. From FaaSKeeper Design To Cloud
  14. Disaggregating
  15. Mapping Storage
  16. ...and 28 more sections

Figures (16)

  • Figure 1: A high-level overview of a FaaS platform.
  • Figure 2: From ZooKeeper servers to functions in FaaSKeeper.
  • Figure 3: Workflow for designing a serverless service. The decoupled compute and storage are connected to cloud services, and later optimized for cost and performance (Section \ref{['sec:faaskeeper-optimizations']}).
  • Figure 4: Cost and performance of storage in the AWS cloud. Python benchmarks executed on an EC2 instance.
  • Figure 5: ZooKeeper utilization in HBase running YCSB.
  • ...and 11 more figures