Debiasing Architectural Decision-Making: An Experiment With Students and Practitioners
Klara Borowa, Rodrigo Rebouças de Almeida, Marion Wiese
TL;DR
The paper investigates cognitive biases in architectural decision-making (ADM) and evaluates a debiasing workshop across students and practitioners. Using a matched-pair design with think-aloud data, it shows the workshop reduces anchoring and optimism bias while increasing the use of debiasing techniques. Practitioners, who exhibit higher baseline biases, benefit more from the intervention, and the study provides actionable guidance for educators and researchers. The findings support deploying structured debiasing training in software architecture education and industry practice to improve ADM outcomes and reduce bias-driven suboptimal decisions.
Abstract
Cognitive biases are predictable, systematic errors in human reasoning. They influence decision-making in various areas, including architectural decision-making, where architects face many choices. For example, anchoring can cause architects to unconsciously prefer the first architectural solution that they came up with, without considering any solution alternatives. Prior research suggests that training individuals in debiasing techniques during a practical workshop can help reduce the impact of biases. The goal of this study was to design and evaluate a debiasing workshop with individuals at various stages of their professional careers. To test the workshop's effectiveness, we performed an experiment with 16 students and 20 practitioners, split into control and workshop group pairs. We recorded and analyzed their think-aloud discussions about improving the architectures of systems they collaborated on. The workshop improved the participants' argumentation when discussing architectural decisions and increased the use of debiasing techniques taught during the workshop. This led to the successful reduction of the researched biases' occurrences. In particular, anchoring and optimism bias occurrences decreased significantly. We also found that practitioners were more susceptible to cognitive biases than students, so the workshop had a more substantial impact on practitioners. We assume that the practitioners' attachment to their systems may be the cause of their susceptibility to biases. Finally, we identified factors that may reduce the effectiveness of the debiasing workshop. On that basis, we prepared a set of teaching suggestions for educators. Overall, we recommend using this workshop to educate both students and experienced practitioners about the typical harmful influences of cognitive bias on architectural decisions and how to avoid them.