Saarthi: An End-to-End Intelligent Platform for Optimising Distributed Serverless Workloads
Siddharth Agarwal, Maria A. Rodriguez, Rajkumar Buyya
TL;DR
Saarthi tackles inefficiencies in FaaS resource management by introducing an input-aware, end-to-end platform that couples per-request predictions with a hybrid orchestrator and a multi-objective ILP optimiser to balance throughput and cost while maintaining SLA through a fault-tolerant redundancy layer. The approach integrates an online Random Forest Regression predictor, an Adaptive Request Balancer with a $G/G/c/K$ queue, a global ILP optimisation engine, and a proactive redundancy mechanism, all implemented atop OpenFaaS/Kubernetes. Empirical results show up to 1.45x throughput improvements and up to 1.84x cost reductions with SLA satisfaction around 98% across diverse workloads, with modest overheads on the critical path. By automating resource configuration, per-request routing, and deployment decisions, Saarthi reduces resource fragmentation and developer burden, enabling more predictable latency and scalable distribution of serverless workloads.
Abstract
FaaS offers significant advantages with its infrastructure abstraction, on-demand execution, and attractive no idle resource pricing for modern cloud applications. Despite these benefits, challenges such as startup latencies, static configurations, sub-optimal resource allocation and scheduling still exist due to coupled resource offering and workload-agnostic generic scheduling behaviour. These issues often lead to inconsistent function performance and unexpected operational costs for users and service providers. This paper introduces Saarthi, a novel, end-to-end serverless framework that intelligently manages the dynamic resource needs of function workloads, representing a significant step toward self-driving serverless platforms. Unlike platforms that rely on static resource configurations, Saarthi is input-aware, allowing it to intelligently anticipate resource requirements based on the characteristics of an incoming request payload. This input-driven approach reinforces function right-sizing and enables smart request orchestration across available function configurations. Saarthi further integrates a proactive fault-tolerant redundancy mechanism and employs a multi-objective Integer Linear Programming (ILP) model to maintain an optimal function quantity. This optimisation aims to maximise system throughput while simultaneously reducing overall operational costs. We validate the effectiveness of Saarthi by implementing it as a framework atop OpenFaaS. Our results demonstrate Saarthi's ability to achieve up to 1.45x better throughput, 1.84x reduced costs, while maintaining up to 98.3% service level targets with an overhead of up to 0.2 seconds as compared to the baseline OpenFaaS.