Minos: Exploiting Cloud Performance Variation with Function-as-a-Service Instance Selection

TL;DR

Problem: FaaS platforms exhibit per-instance performance variation due to shared infrastructure, causing longer execution and higher costs for slow instances. Method: Minos benchmarks new function instances at cold start and terminates underperformers to keep a pool of fast ones, using an elysium threshold to decide locally; an initial pre-test sets this threshold; benchmarking can run in parallel with setup steps. Contributions: a proof-of-concept on Google Cloud Functions, a concrete termination policy with the $c_{\textsf{exec}}$ cost model, and an evaluation showing up to $13\%$ improvement in resource-intensive steps and up to $4\%$ overall speedup and $0.9\%$ cost reduction. Significance: demonstrates that exploiting platform variability can yield performance and cost benefits, while prompting discussion of fairer virtualization and live-threshold updates.

Abstract

Serverless Function-as-a-Service (FaaS) is a popular cloud paradigm to quickly and cheaply implement complex applications. Because the function instances cloud providers start to execute user code run on shared infrastructure, their performance can vary. From a user perspective, slower instances not only take longer to complete, but also increase cost due to the pay-per-use model of FaaS services where execution duration is billed with microsecond accuracy. In this paper, we present Minos, a system to take advantage of this performance variation by intentionally terminating instances that are slow. Fast instances are not terminated, so that they can be re-used for subsequent invocations. One use case for this are data processing and machine learning workflows, which often download files as a first step, during which Minos can run a short benchmark. Only if the benchmark passes, the main part of the function is actually executed. Otherwise, the request is re-queued and the instance crashes itself, so that the platform has to assign the request to another (potentially faster) instance. In our experiments, this leads to a speedup of up to 13% in the resource intensive part of a data processing workflow, resulting in up to 4% faster overall performance (and consequently 4% cheaper prices). Longer and complex workflows lead to increased savings, as the pool of fast instances is re-used more often. For platforms exhibiting this behavior, users get better performance and save money by wasting more of the platforms resources.

Figures (7)

• Figure 1: FaaS platforms start new function instances on shared worker nodes. While users can not influence which worker node will be used, underutilized nodes offer two benefits: faster execution times and lower cost. Using the Minos system, newly started instances check if they are running on a node with low utilization. If not, they terminate themselves by crashing. This leads to a pool of better-performing instances that are re-used for subsequent invocations, leading to a compound performance increase.
• Figure 2: Overview of the process inside a function instance. The first step of the invocation, called prepare, is always executed. If the invocation is a cold start, a benchmark is executed in parallel. Minos then judges whether the result is better than the elysium threshold, in which case nothing happens (and the instance will be re-used in the future). If the instance does not pass, further execution is interrupted, the invocation is re-queued, and the instance is terminated.
• Figure 3: The total cost of a workflow comprises the cost per millisecond of execution ($c_{\textsf{exec}}$) of all terminated ($d_{\textsf{term}}$), successful ($d_{\textsf{pass}}$) benchmarks and the re-used instances ($d_{\textsf{reuse}}$), as well as the cost per invocation ($c_{\textsf{inv}}$) per terminated, passed, and re-used function.
• Figure 4: The linear regression step was, on average, faster every day the experiment was repeated. The maximum improvement was $>13\%$ on day two, and minimum improvement $4.3\%$ on day three and five. Note that the y-axis is limited to values between 1,000 and 3,000.
• Figure 5: Successful requests per day. Minos improved the amount of successful requests all days except one with a maximum of $7.3\%$ on day one, but reduced the amount by $<1\%$ on day five. Note that the y-axis is limited to values between 4,000 and 5,000.