An Empirical Evaluation of Serverless Cloud Infrastructure for Large-Scale Data Processing

TL;DR

This work provides an end-to-end empirical evaluation of serverless cloud infrastructure for large-scale data processing using the Skyrise framework. It characterizes bursty network performance, storage IOPS scaling, and the translation of resource-level phenomena into application-level query behavior, while detailing cost break-even points for serverless compute and storage versus VM-based deployments. Key findings include substantial network bursting advantages for scan-heavy workloads, S3-based object storage as the most economical scalable option with Express variants delivering lower latency, and clear break-even thresholds that guide when serverless analytics are financially viable. The insights offer practical guidance for designing serverless data pipelines and set benchmarks for future evaluations of disaggregated, serverless data systems.

Abstract

Data processing systems are increasingly deployed in the cloud. While monolithic systems run fully on virtual servers, recent systems embrace cloud infrastructure and utilize the disaggregation of compute and storage to scale them independently. The introduction of serverless compute services, such as AWS Lambda, enables finer-grained and elastic scalability within these systems. Prior work shows the viability of serverless infrastructure for scalable data processing yet also sees limitations due to variable performance and cost overhead, in particular for networking and storage. In this paper, we perform a detailed analysis of the performance and cost characteristics of serverless infrastructure in the data processing context. We base our analysis on a large series of micro-benchmarks across different compute and storage services, as well as end-to-end workloads. To enable our analysis, we propose the Skyrise serverless evaluation platform. For the widely used serverless infrastructure of AWS, our analysis reveals distinct boundaries for performance variability in serverless networks and storage. We further present cost break-even points for serverless compute and storage. These insights provide guidance on when and how serverless infrastructure can be efficiently used for data processing.

Figures (15)

• Figure 1: Architecture of the AWS Lambda FaaS platform with control (dotted) and data (solid) paths.
• Figure 2: Schematic architecture of a serverless storage system with control (dotted) and data (solid) planes.
• Figure 3: Architecture of the Skyrise evaluation framework showing the experiment execution flow from config to plot.
• Figure 4: Execution modes of the Skyrise query engine with serverless functions (upper path) and servers (lower path).
• Figure 5: Function network throughput at 20 ms intervals with short sleep to refill inbound/outbound token buckets.