Universal Workers: A Vision for Eliminating Cold Starts in Serverless Computing

TL;DR

This work tackles the cold-start problem in serverless computing by proposing universal workers that can execute any function with minimal initialization. It leverages the skew in function popularity through locality groups and a three-tier caching hierarchy to approximate a universal executor, reducing cold-start latency across diverse workloads. The approach includes a graph-based locality clustering method and Linux-based cache mechanisms, with evaluation on four production traces showing substantial cache-hit rates and practical memory considerations. If deployed, this could significantly improve FaaS scalability and throughput, with open-source implementation planned for broader adoption.

Abstract

Serverless computing enables developers to deploy code without managing infrastructure, but suffers from cold start overhead when initializing new function instances. Existing solutions such as "keep-alive" or "pre-warming" are costly and unreliable under bursty workloads. We propose universal workers, which are computational units capable of executing any function with minimal initialization overhead. Based on an analysis of production workload traces, our key insight is that requests in Function-as-a-Service (FaaS) platforms show a highly skewed distribution, with most requests invoking a small subset of functions. We exploit this observation to approximate universal workers through locality groups and three-tier caching (handler, install, import). With this work, we aim to enable more efficient and scalable FaaS platforms capable of handling diverse workloads with minimal initialization overhead.

Figures (5)

• Figure 1: Latency breakdown of a function running the linpack benchmark (n=1000) on OpenLambda.
• Figure 2: Skew in function popularity. A small number of functions account for the majority of requests.
• Figure 3: Locality group assignment. Functions are distributed across locality groups that have a pool of workers.
• Figure 4: Import cache. A tree-based caching system where each node represents a sleeping process with pre-imported packages.
• Figure 5: Cache hit rates. Even small cache sizes achieve high rates due to skewed function popularity, demonstrating the feasibility of universal workers.