Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Umbilical Choir: Automated Live Testing for Edge-To-Cloud FaaS Applications

Mohammadreza Malekabbasi, Tobias Pfandzelter, David Bermbach

TL;DR

The paper addresses the difficulty of maintaining QoS during frequent edge-to-cloud releases by introducing Umbilical Choir, a provider-agnostic live-testing framework for serverless edge-to-cloud applications. It presents a tree-based architecture with Release Managers, Agents, and a proxy function to enable A/B testing, canary releases with geo-aware strategies, dark launches, and gradual rollouts across geo-distributed FaaS platforms. The contributions include a detailed architectural design, four geo-aware rollout strategies, an open-source Go prototype supporting major cloud and edge FaaS, and an evaluation consisting of a latency-overhead experiment and a realistic multi-stage release scenario. Results demonstrate the framework’s ability to automate complex, multi-node release plans with acceptable overhead, highlighting its practical potential for reducing performance regressions in edge-to-cloud deployments while noting future work to integrate native provider live-testing features and broader runtime support.

Abstract

Application users react negatively to performance regressions or availability issues across software releases. To address this, modern cloud-based applications with their multiple daily releases rely on live testing techniques such as A/B testing or canary releases. In edge-to-cloud applications, however, which have similar problems, developers currently still have to hard-code custom live testing tooling as there is no general framework for edge-to-cloud live testing. With Umbilical Choir, we partially close this gap for serverless edge-to-cloud applications. Umbilical Choir is compatible with all Function-as-a-Service platforms and (extensively) supports various live testing techniques, including canary releases with various geo-aware strategies, A/B testing, and gradual roll-outs. We evaluate Umbilical Choir through a complex release scenario showcasing various live testing techniques in a mixed edge-cloud deployments and discuss different geo-aware strategies.

Umbilical Choir: Automated Live Testing for Edge-To-Cloud FaaS Applications

TL;DR

The paper addresses the difficulty of maintaining QoS during frequent edge-to-cloud releases by introducing Umbilical Choir, a provider-agnostic live-testing framework for serverless edge-to-cloud applications. It presents a tree-based architecture with Release Managers, Agents, and a proxy function to enable A/B testing, canary releases with geo-aware strategies, dark launches, and gradual rollouts across geo-distributed FaaS platforms. The contributions include a detailed architectural design, four geo-aware rollout strategies, an open-source Go prototype supporting major cloud and edge FaaS, and an evaluation consisting of a latency-overhead experiment and a realistic multi-stage release scenario. Results demonstrate the framework’s ability to automate complex, multi-node release plans with acceptable overhead, highlighting its practical potential for reducing performance regressions in edge-to-cloud deployments while noting future work to integrate native provider live-testing features and broader runtime support.

Abstract

Application users react negatively to performance regressions or availability issues across software releases. To address this, modern cloud-based applications with their multiple daily releases rely on live testing techniques such as A/B testing or canary releases. In edge-to-cloud applications, however, which have similar problems, developers currently still have to hard-code custom live testing tooling as there is no general framework for edge-to-cloud live testing. With Umbilical Choir, we partially close this gap for serverless edge-to-cloud applications. Umbilical Choir is compatible with all Function-as-a-Service platforms and (extensively) supports various live testing techniques, including canary releases with various geo-aware strategies, A/B testing, and gradual roll-outs. We evaluate Umbilical Choir through a complex release scenario showcasing various live testing techniques in a mixed edge-cloud deployments and discuss different geo-aware strategies.

Paper Structure

This paper contains 21 sections, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Serverless Edge-to-Cloud Computing
  4. Multi-Phase Live Testing
  5. Testing (Serverless) Edge-to-Cloud Applications
  6. Design of Umbilical Choir
  7. Architecture
  8. Components
  9. Release Manager
  10. Agent
  11. Proxy Function
  12. Defining Release Strategies
  13. Gradual Roll-out Strategies
  14. Evaluation
  15. Proof-of-Concept Prototype
  16. ...and 6 more sections

Figures (4)

  • Figure 1: High-level Architecture of Umbilical Choir.
  • Figure 2: Agent Interaction Architecture During a Live Test
  • Figure 3: Response times for direct function calls and calls mediated by a proxy function, evaluated on Google Cloud Functions and Raspberry/tinyFaaS
  • Figure 4: A realistic release strategy scenario demonstrating canary tests, A/B tests, and gradual roll-out stages while two parallel clients keep calling the function with a constant rate. The figure shows the client-observable latency over time, with one client calling the local edge node and one calling the cloud function.