Comparison of Microservice Call Rate Predictions for Replication in the Cloud

TL;DR

This work tackles provisioning of microservice replicas under stochastic user demand by predicting microservice call rates from processing time. It compares linear regression, multilayer perceptron, and gradient boosting regression on Alibaba trace data, finding that gradient boosting offers the best MAE/MAPE balance while remaining computationally efficient, and can estimate replica counts with little to no prediction error. The study presents a DataCloud architecture with DEF-PIPE, SIM-PIPE, ADA-PIPE, and DEP-PIPE to support end-to-end replication planning and deployment, highlighting the potential for integrating these predictions into Kubernetes autoscaling. Overall, the GBR approach delivers practical gains for cloud-based microservice autoscaling and replication planning, enabling responsive and near-accurate provisioning under dynamic workloads.

Abstract

Today, many users deploy their microservice-based applications with various interconnections on a cluster of Cloud machines, subject to stochastic changes due to dynamic user requirements. To address this problem, we compare three machine learning (ML) models for predicting the microservice call rates based on the microservice times and aiming at estimating the scalability requirements. We apply the linear regression (LR), multilayer perception (MLP), and gradient boosting regression (GBR) models on the Alibaba microservice traces. The prediction results reveal that the LR model reaches a lower training time than the GBR and MLP models. However, the GBR reduces the mean absolute error and the mean absolute percentage error compared to LR and MLP models. Moreover, the prediction results show that the required number of replicas for each microservice by the gradient boosting model is close to the actual test data without any prediction.

Figures (5)

• Figure 1: MLP neural network architecture.
• Figure 2: $ADA-PIPE$ architecture.
• Figure 3: GBR and MLP losses.
• Figure 4: Distribution and correlation of $\mathtt{MT}_{i,j}$ and $\mathtt{MCR}_{\mathcal{P}{}i}$ in Alibaba microservices dataset.
• Figure 5: ML model fitting to the test dataset.