The Role of Cognitive Abilities in Requirements Inspection: Comparing UML and Textual Representations
Giovanna Broccia, Sira Vegas, Alessio Ferrari
TL;DR
The paper investigates how cognitive abilities—specifically working memory capacity and mental rotation—modulate the effectiveness of UML sequence diagrams in requirements inspection. Using a crossover design with 38 participants, the authors compare text-only requirements to text-plus-UML across two tasks, measuring both issue detection (F1-score) and justification quality (Accuracy Why). A key finding is a significant three-way interaction showing UML can decrease detection performance for some cognitive profiles while enhancing justification when both cognitive abilities are strong, highlighting a cognitive-fit effect and the risk of overload with multimodal representations. The study contributes to understanding when UML aids or hinders inspection and suggests adapting representations to individual cognitive profiles, with implications for training and tool support. Limitations include limited variability in rotation ability and the student sample, guiding future work toward more diverse participants and adaptive inspection environments.
Abstract
The representation of requirements plays a critical role in the accuracy of requirements inspection. While visual representations, such as UML diagrams, are widely used alongside text-based requirements, their effectiveness in supporting inspection is still debated. Cognitive abilities, such as working memory and mental rotation skills, may also influence inspection accuracy. This study aims to evaluate whether the use of UML sequence diagrams alongside text-based requirements improves the accuracy of requirements inspection compared to text-based requirements alone and to explore whether cognitive abilities are associated with differences in performance across the two treatments (text vs text with UML support). We conducted a crossover experiment with 38 participants to assess the accuracy of requirements inspection under the two treatments in terms of issues found and justifications provided. Linear mixed-effects and generalized linear models were used to analyse the effects of treatment, period, sequence, and cognitive abilities. The results indicate a significant three-way interaction between representation type, working memory capacity, and mental rotation ability. This finding suggests that the effectiveness of UML support is not uniform across individuals: participants with high scores in both cognitive abilities experienced reduced performance when using UML for violation detection. Conversely, the same cognitive profile was associated with improved justification accuracy under UML-aided inspection, indicating that higher cognitive abilities may support deeper reasoning processes when dealing with multi-modal information, i.e., diagrams and text.