Towards Assessing Spread in Sets of Software Architecture Designs

TL;DR

This work tackles the challenge of evaluating how broadly sets of software-architecture design alternatives cover the design space when generated by multi-objective optimization. It introduces Maximum Architectural Spread (MAS), an architectural-space analogue of the objective-space spread, by encoding architectures as sequences of refactoring transformations and measuring pairwise distances with Levenshtein-based sequence metrics. The authors demonstrate MAS on multiple datasets (e.g., ST+ with PCM architectures, Train Ticket, CoCoME) and across several evolutionary algorithms, showing that MAS correlates with the traditional MS indicator while providing additional architectural insight into diversity and the effects of optimization settings. The approach enables designers to assess and compare optimization configurations from an architectural viewpoint, with plans to explore additional distance metrics and complementary indicators for uniformity and convergence in the architectural space.

Abstract

Several approaches have recently used automated techniques to generate architecture design alternatives by means of optimization techniques. These approaches aim at improving an initial architecture with respect to quality aspects, such as performance, reliability, or maintainability. In this context, each optimization experiment usually produces a different set of architecture alternatives that is characterized by specific settings. As a consequence, the designer is left with the task of comparing such sets to identify the settings that lead to better solution sets for the problem. To assess the quality of solution sets, multi-objective optimization commonly relies on quality indicators. Among these, the quality indicator for the maximum spread estimates the diversity of the generated alternatives, providing a measure of how much of the solution space has been explored. However, the maximum spread indicator is computed only on the objective space and does not consider architectural information (e.g., components structure, design decisions) from the architectural space. In this paper, we propose a quality indicator for the spread that assesses the diversity of alternatives by taking into account architectural features. To compute the spread, we rely on a notion of distance between alternatives according to the way they were generated during the optimization. We demonstrate how our architectural quality indicator can be applied to a dataset from the literature.

Figures (2)

• Figure 1: Relationship between the architectural and objective spaces. The architectural space is represented as search tree rooted at the initial architecture ($A_0$).
• Figure 2: Comparison of the architectural and objective spaces for the two search algorithms in ST+, along with computed $\mathit{MAS}$ and MS.