Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

KUBEDIRECT: Unleashing the Full Power of the Cluster Manager for Serverless Computing

Sheng Qi, Zhiquan Zhang, Xuanzhe Liu, Xin Jin

TL;DR

Kubedirect retrofits Kubernetes to handle bursty FaaS workloads by bypassing the API Server with direct, delta-based messaging along the narrow waist of controllers. It introduces a hierarchical write-back cache with hard and soft invalidations and a handshake protocol to maintain end-to-end correctness without centralized coordination. Dynamic materialization decouples dynamic from static API attributes, enabling compact on-wire messages (~64B per object) and transparent processing. Empirical results show $26.7\times$ end-to-end latency reductions over Knative and competitive performance with Dirigent, while maintaining backward compatibility and extensibility for Kubernetes ecosystems.

Abstract

FaaS platforms rely on cluster managers like Kubernetes for resource management. Kubernetes is popular due to its state-centric APIs that decouple the control plane into modular controllers. However, to scale out a burst of FaaS instances, message passing becomes the primary bottleneck as controllers have to exchange extensive state through the API Server. Existing solutions opt for a clean-slate redesign of cluster managers, but at the expense of compatibility with existing ecosystem and substantial engineering effort. We present KUBEDIRECT, a Kubernetes-based cluster manager for FaaS. We find that there exists a common narrow waist across FaaS platform that allows us to achieve both efficiency and external compatibility. Our insight is that the sequential structure of the narrow waist obviates the need for a single source of truth, allowing us to bypass the API Server and perform direct message passing for efficiency. However, our approach introduces a set of ephemeral states across controllers, making it challenging to enforce end-to-end semantics due to the absence of centralized coordination. KUBEDIRECT employs a novel state management scheme that leverages the narrow waist as a hierarchical write-back cache, ensuring consistency and convergence to the desired state. KUBEDIRECT can seamlessly integrate with Kubernetes, adding ~150 LoC per controller. Experiments show that KUBEDIRECT reduces serving latency by 26.7x over Knative, and has similar performance as the state-of-the-art clean-slate platform Dirigent.

KUBEDIRECT: Unleashing the Full Power of the Cluster Manager for Serverless Computing

TL;DR

Kubedirect retrofits Kubernetes to handle bursty FaaS workloads by bypassing the API Server with direct, delta-based messaging along the narrow waist of controllers. It introduces a hierarchical write-back cache with hard and soft invalidations and a handshake protocol to maintain end-to-end correctness without centralized coordination. Dynamic materialization decouples dynamic from static API attributes, enabling compact on-wire messages (~64B per object) and transparent processing. Empirical results show end-to-end latency reductions over Knative and competitive performance with Dirigent, while maintaining backward compatibility and extensibility for Kubernetes ecosystems.

Abstract

FaaS platforms rely on cluster managers like Kubernetes for resource management. Kubernetes is popular due to its state-centric APIs that decouple the control plane into modular controllers. However, to scale out a burst of FaaS instances, message passing becomes the primary bottleneck as controllers have to exchange extensive state through the API Server. Existing solutions opt for a clean-slate redesign of cluster managers, but at the expense of compatibility with existing ecosystem and substantial engineering effort. We present KUBEDIRECT, a Kubernetes-based cluster manager for FaaS. We find that there exists a common narrow waist across FaaS platform that allows us to achieve both efficiency and external compatibility. Our insight is that the sequential structure of the narrow waist obviates the need for a single source of truth, allowing us to bypass the API Server and perform direct message passing for efficiency. However, our approach introduces a set of ephemeral states across controllers, making it challenging to enforce end-to-end semantics due to the absence of centralized coordination. KUBEDIRECT employs a novel state management scheme that leverages the narrow waist as a hierarchical write-back cache, ensuring consistency and convergence to the desired state. KUBEDIRECT can seamlessly integrate with Kubernetes, adding ~150 LoC per controller. Experiments show that KUBEDIRECT reduces serving latency by 26.7x over Knative, and has similar performance as the state-of-the-art clean-slate platform Dirigent.
Paper Structure (22 sections, 15 figures)

This paper contains 22 sections, 15 figures.

Table of Contents

  1. Introduction
  2. Background and Motivation
  3. Kubernetes Basics
  4. The Message Passing Bottleneck
  5. Opportunities for Efficient Message Passing
  6. Challenges for State Management
  7. Kubedirect Overview
  8. Architecture
  9. Dynamic Materialization
  10. State Management
  11. Problem Analysis
  12. Hierarchical Write-Back Cache
  13. Instance Lifecycle Management
  14. Correctness
  15. Implementation
  16. ...and 7 more sections

Figures (15)

  • Figure 1: The narrow waist of Kubernetes-based FaaS platform and the scaling critical path. We consider scaling $N$ Pods for $K$ ReplicaSets in an $M$-node cluster.
  • Figure 2: Architecture of Kubernetes-based FaaS platform. We highlight the narrow waist in orange. 1 to 5 are from Figure \ref{['fig:bg:k8s-narrow-waist']} and are indirect calls via the API Server.
  • Figure 3: The gap between Kubernetes and serverless.
  • Figure 4: Kubedirect's (Kd) integration with Kubernetes (K8s) controllers. The added steps are labeled with asterisks.
  • Figure 5: The minimal message format in Kubedirect.
  • ...and 10 more figures