Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Gaia: Hybrid Hardware Acceleration for Serverless AI in the 3D Compute Continuum

Maximilian Reisecker, Cynthia Marcelino, Thomas Pusztai, Stefan Nastic

TL;DR

This paper tackles the challenge of allocating hardware acceleration for serverless AI across the Edge-Cloud-Space 3D Continuum, where resource availability and connectivity are highly dynamic. It introduces Gaia, a GPU-as-a-Service model composed of an Execution Mode Identifier for pre-deployment hints and a Dynamic Function Runtime for runtime adaptation, both guided by SLOs and telemetry. The authors demonstrate up to a $95\%$ reduction in end-to-end latency compared with CPU-only execution and provide an open-source prototype with a Python-based dynamic runtime and a Go-based static analyzer. The work enables SLO-aware, cost-efficient acceleration across heterogeneous environments by moving hardware decisions from developers to the platform and continuously re-evaluating them as conditions change.

Abstract

Serverless computing offers elastic scaling and pay-per-use execution, making it well-suited for AI workloads. As these workloads run in heterogeneous environments such as the Edge-Cloud-Space 3D Continuum, they often require intensive parallel computation, which GPUs can perform far more efficiently than CPUs. However, current platforms struggle to manage hardware acceleration effectively, as static user-device assignments fail to ensure SLO compliance under varying loads or placements, and one-time dynamic selections often lead to suboptimal or cost-inefficient configurations. To address these issues, we present Gaia, a GPU-as-a-service model and architecture that makes hardware acceleration a platform concern. Gaia combines (i) a lightweight Execution Mode Identifier that inspects function code at deploy time to emit one of four execution modes, and a Dynamic Function Runtime that continuously reevaluates user-defined SLOs to promote or demote between CPU- and GPU backends. Our evaluation shows that it seamlessly selects the best hardware acceleration for the workload, reducing end-to-end latency by up to 95%. These results indicate that Gaia enables SLO-aware, cost-efficient acceleration for serverless AI across heterogeneous environments.

Gaia: Hybrid Hardware Acceleration for Serverless AI in the 3D Compute Continuum

TL;DR

This paper tackles the challenge of allocating hardware acceleration for serverless AI across the Edge-Cloud-Space 3D Continuum, where resource availability and connectivity are highly dynamic. It introduces Gaia, a GPU-as-a-Service model composed of an Execution Mode Identifier for pre-deployment hints and a Dynamic Function Runtime for runtime adaptation, both guided by SLOs and telemetry. The authors demonstrate up to a reduction in end-to-end latency compared with CPU-only execution and provide an open-source prototype with a Python-based dynamic runtime and a Go-based static analyzer. The work enables SLO-aware, cost-efficient acceleration across heterogeneous environments by moving hardware decisions from developers to the platform and continuously re-evaluating them as conditions change.

Abstract

Serverless computing offers elastic scaling and pay-per-use execution, making it well-suited for AI workloads. As these workloads run in heterogeneous environments such as the Edge-Cloud-Space 3D Continuum, they often require intensive parallel computation, which GPUs can perform far more efficiently than CPUs. However, current platforms struggle to manage hardware acceleration effectively, as static user-device assignments fail to ensure SLO compliance under varying loads or placements, and one-time dynamic selections often lead to suboptimal or cost-inefficient configurations. To address these issues, we present Gaia, a GPU-as-a-service model and architecture that makes hardware acceleration a platform concern. Gaia combines (i) a lightweight Execution Mode Identifier that inspects function code at deploy time to emit one of four execution modes, and a Dynamic Function Runtime that continuously reevaluates user-defined SLOs to promote or demote between CPU- and GPU backends. Our evaluation shows that it seamlessly selects the best hardware acceleration for the workload, reducing end-to-end latency by up to 95%. These results indicate that Gaia enables SLO-aware, cost-efficient acceleration for serverless AI across heterogeneous environments.

Paper Structure

This paper contains 38 sections, 7 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Illustrative Scenario and Research Challenges
  3. Illustrative Scenario
  4. Research Challenges
  5. Gaia Design Principles and Architecture
  6. Gaia Design Principles
  7. Hardware-agnostic
  8. Zero Developer Friction
  9. Intelligent Start & Dynamic Switching.
  10. Observability by Design.
  11. Gaia Architecture
  12. Gaia Components
  13. Code Analyzer
  14. Build and Deploy
  15. Controller
  16. ...and 23 more sections

Figures (7)

  • Figure 1: Deforestation Illustrative Scenario with Serverless Functions Execution in the 3D Compute Continuum
  • Figure 2: Gaia Architecture
  • Figure 3: Gaia Dynamic Runtime Function Adaptation
  • Figure 4: Serverless Function Execution Latency with Dynamic Function Runtime Reconfiguration
  • Figure 5: Matrix multiplication: Gaia promotes to GPU once the SLO is hit, producing a clear step-down in latency and cost relative to CPU.
  • ...and 2 more figures