A limited technical background is sufficient for attack-defense tree acceptability

TL;DR

The paper examines whether attack-defense trees (ADTs) are acceptable to stakeholders with limited technical backgrounds. Using MEM/TAM-inspired measures in a between-subjects design with $n=102$ participants (LT vs HT) across small and large study components, the authors assess actual effectiveness, perceived usefulness, ease of use, and intention to use, along with creative ADT design. They find strong evidence that a limited CS background suffices for ADT acceptability, with comparable performance and perceptions to highly technical users, and they document similar creativity and tree quality across groups. The study highlights short, accessible training as sufficient to democratize ADT usage and provides open data, materials, and tools to foster further research and adoption in diverse threat-modeling teams.

Abstract

Attack-defense trees (ADTs) are a prominent graphical threat modeling method that is highly recommended for analyzing and communicating security-related information. Despite this, existing empirical studies of attack trees have established their acceptability only for users with highly technical (computer science) backgrounds while raising questions about their suitability for threat modeling stakeholders with a limited technical background. Our research addresses this gap by investigating the impact of the users' technical background on ADT acceptability in an empirical study. Our Method Evaluation Model-based study consisted of n = 102 participants (53 with a strong computer science background and 49 with a limited computer science background) who were asked to complete a series of ADT-related tasks. By analyzing their responses and comparing the results, we reveal that a very limited technical background is sufficient for ADT acceptability. This finding underscores attack trees' viability as a threat modeling method.

Figures (11)

• Figure 1: An example of an ADT (the second ADT in our small study) from sunCyberAttackRisksAnalysis2018.
• Figure 2: The Method Evaluation Model (MEM) moodyMethodEvaluationModel2003 with our study hypotheses placed in context.
• Figure 3: Distribution of participants across the treatment groups and studies.
• Figure 4: Comparison of the average scores across check-for-understanding questions.
• Figure 5: Comparison on creating ADTs from a written description.