Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Experimental evaluation of architectural software performance design patterns in microservices

Willem Meijer, Catia Trubiani, Aldeida Aleti

TL;DR

This work addresses the challenge of quantifying how architectural design patterns in microservice systems affect performance under heterogeneous workloads. It builds a realistic cloud-based testbed and implements three patterns—Gateway Aggregation, Gateway Offloading, and Pipes and Filters—to collect real performance measurements and compare them against model-based predictions from prior queuing-network analyses. The study finds strong global agreement between theory and experiment in behavior, but notable deviations in absolute values, particularly for CPU utilization and bottleneck transitions, highlighting the necessity of empirical validation. Overall, the results advocate a continuous feedback loop between theoretical models and experimental data to improve the accuracy and reliability of performance predictions for microservice design patterns, with practical guidance for software architects and performance engineers.

Abstract

Microservice architectures and design patterns enhance the development of large-scale applications by promoting flexibility. Industrial practitioners perceive the importance of applying architectural patterns but they struggle to quantify their impact on system quality requirements. Our research aims to quantify the effect of design patterns on system performance metrics, e.g., service latency and resource utilization, even more so when the patterns operate in real-world environments subject to heterogeneous workloads. We built a cloud infrastructure to host a well-established benchmark system that represents our test bed, complemented by the implementation of three design patterns: Gateway Aggregation, Gateway Offloading, Pipe and Filters. Real performance measurements are collected and compared with model-based predictions that we derived as part of our previous research, thus further consolidating the actual impact of these patterns. Our results demonstrate that, despite the difficulty to parameterize our benchmark system, model-based predictions are in line with real experimentation, since the performance behaviors of patterns, e.g., bottleneck switches, are mostly preserved. In summary, this is the first work that experimentally demonstrates the performance behavior of microservices-based architectural patterns. Results highlight the complexity of evaluating the performance of design patterns and emphasize the need for complementing theoretical models with empirical data.

Experimental evaluation of architectural software performance design patterns in microservices

TL;DR

This work addresses the challenge of quantifying how architectural design patterns in microservice systems affect performance under heterogeneous workloads. It builds a realistic cloud-based testbed and implements three patterns—Gateway Aggregation, Gateway Offloading, and Pipes and Filters—to collect real performance measurements and compare them against model-based predictions from prior queuing-network analyses. The study finds strong global agreement between theory and experiment in behavior, but notable deviations in absolute values, particularly for CPU utilization and bottleneck transitions, highlighting the necessity of empirical validation. Overall, the results advocate a continuous feedback loop between theoretical models and experimental data to improve the accuracy and reliability of performance predictions for microservice design patterns, with practical guidance for software architects and performance engineers.

Abstract

Microservice architectures and design patterns enhance the development of large-scale applications by promoting flexibility. Industrial practitioners perceive the importance of applying architectural patterns but they struggle to quantify their impact on system quality requirements. Our research aims to quantify the effect of design patterns on system performance metrics, e.g., service latency and resource utilization, even more so when the patterns operate in real-world environments subject to heterogeneous workloads. We built a cloud infrastructure to host a well-established benchmark system that represents our test bed, complemented by the implementation of three design patterns: Gateway Aggregation, Gateway Offloading, Pipe and Filters. Real performance measurements are collected and compared with model-based predictions that we derived as part of our previous research, thus further consolidating the actual impact of these patterns. Our results demonstrate that, despite the difficulty to parameterize our benchmark system, model-based predictions are in line with real experimentation, since the performance behaviors of patterns, e.g., bottleneck switches, are mostly preserved. In summary, this is the first work that experimentally demonstrates the performance behavior of microservices-based architectural patterns. Results highlight the complexity of evaluating the performance of design patterns and emphasize the need for complementing theoretical models with empirical data.
Paper Structure (27 sections, 7 figures, 6 tables)

This paper contains 27 sections, 7 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Design of experiments
  4. Theoretical performance models
  5. Cloud infrastructure
  6. Experimental settings
  7. Microservice modeling
  8. Workload specification
  9. Specification of design patterns
  10. Gateway aggregation
  11. Gateway offloading
  12. Pipes and filters
  13. Evaluation of analysis results
  14. Metrics of interest
  15. Design patterns results
  16. ...and 12 more sections

Figures (7)

  • Figure 1: Overview of the used cluster and experiment topology.
  • Figure 2: Model of the gateway aggregation pattern
  • Figure 3: Model of the gateway offloading pattern
  • Figure 4: Models of the pipes and filters pattern variants
  • Figure 5: Overview of the theoretical pinciroli_performance_2023 and experimental results in terms of response time and CPU utilization of the gateway aggregation pattern.
  • ...and 2 more figures