Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

How Do Microservice API Patterns Impact Understandability? A Controlled Experiment

Justus Bogner, Pawel Wójcik, Olaf Zimmermann

TL;DR

The correlations between performance and demographics seem to suggest that certain patterns may introduce additional complexity; people experienced with MAPs will profit more from their effects, which has important implications for training and education around MAPs and other patterns.

Abstract

Microservices expose their functionality via remote Application Programming Interfaces (APIs), e.g., based on HTTP or asynchronous messaging technology. To solve recurring problems in this design space, Microservice API Patterns (MAPs) have emerged to capture the collective experience of the API design community. At present, there is a lack of empirical evidence for the effectiveness of these patterns, e.g., how they impact understandability and API usability. We therefore conducted a controlled experiment with 6 microservice patterns to evaluate their impact on understandability with 65 diverse participants. Additionally, we wanted to study how demographics like years of professional experience or experience with MAPs influence the effects of the patterns. Per pattern, we constructed two API examples, each in a pattern version "P" and a functionally equivalent non-pattern version "N" (24 in total). Based on a crossover design, participants had to answer comprehension questions, while we measured the time. For five of the six patterns, we identified a significant positive impact on understandability, i.e., participants answered faster and / or more correctly for "P". However, effect sizes were mostly small, with one pattern showing a medium effect. The correlations between performance and demographics seem to suggest that certain patterns may introduce additional complexity; people experienced with MAPs will profit more from their effects. This has important implications for training and education around MAPs and other patterns.

How Do Microservice API Patterns Impact Understandability? A Controlled Experiment

TL;DR

The correlations between performance and demographics seem to suggest that certain patterns may introduce additional complexity; people experienced with MAPs will profit more from their effects, which has important implications for training and education around MAPs and other patterns.

Abstract

Microservices expose their functionality via remote Application Programming Interfaces (APIs), e.g., based on HTTP or asynchronous messaging technology. To solve recurring problems in this design space, Microservice API Patterns (MAPs) have emerged to capture the collective experience of the API design community. At present, there is a lack of empirical evidence for the effectiveness of these patterns, e.g., how they impact understandability and API usability. We therefore conducted a controlled experiment with 6 microservice patterns to evaluate their impact on understandability with 65 diverse participants. Additionally, we wanted to study how demographics like years of professional experience or experience with MAPs influence the effects of the patterns. Per pattern, we constructed two API examples, each in a pattern version "P" and a functionally equivalent non-pattern version "N" (24 in total). Based on a crossover design, participants had to answer comprehension questions, while we measured the time. For five of the six patterns, we identified a significant positive impact on understandability, i.e., participants answered faster and / or more correctly for "P". However, effect sizes were mostly small, with one pattern showing a medium effect. The correlations between performance and demographics seem to suggest that certain patterns may introduce additional complexity; people experienced with MAPs will profit more from their effects. This has important implications for training and education around MAPs and other patterns.
Paper Structure (17 sections, 1 equation, 3 figures, 6 tables)

This paper contains 17 sections, 1 equation, 3 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Microservice API Patterns (MAPs)
  4. Related Work
  5. Study Design
  6. Research Questions
  7. Participants and Sampling
  8. Experiment Objects and Tasks
  9. Experiment Variables and Hypotheses
  10. Experiment Design and Execution
  11. Experiment Analysis
  12. Results
  13. Impact of Patterns (RQ1)
  14. Influence of Demographic Data (RQ2)
  15. Discussion
  16. ...and 2 more sections

Figures (3)

  • Figure 1: Error Report 1 as an example of a single choice task (pattern version P on the left, non-pattern version N on the right, correct answers checked)
  • Figure 2: Request Bundle 2 as an example of a free-text task (pattern version P on the left, non-pattern version N on the right, correct answers filled in)
  • Figure 3: Boxplots of TAU values for all tasks; pattern versions P in blue (dark-gray in b/w print), non-pattern versions N in orange (light-gray in b/w print)