An instrument to measure factors that constitute the socio-technical context of testing experience

TL;DR

This paper investigates how the socio-technical context shapes software testing experience by framing testing as a cooperative practice influenced by tools, tests, and human needs. It proposes a survey instrument to measure ten interrelated factors across organizational, technical, social, and cultural dimensions, grounded in prior qualitative work and a lean literature review. By preregistering hypotheses and sharing the instrument ahead of data collection, the authors promote open science and methodological rigor while aiming to identify which factor configurations best enable testing motivation and adoption. The anticipated contributions include actionable insights for aligning testing tools and practices with developers' needs in diverse organizational contexts, ultimately enhancing testing effectiveness and uptake.

Abstract

We consider testing a cooperative and social practice that is shaped by the tools developers use, the tests they write, and their mindsets and human needs. This work is one part of a project that explores the human- and socio-technical context of testing through the lens of those interwoven elements: test suite and tools as technical infrastructure and collaborative factors and motivation as mindset. Drawing on empirical observations of previous work, this survey examines how these factors relate to each other. We want to understand which combination of factors can help developers strive and make the most of their ambitions to leverage the potential that software testing practices have. In this report, we construct a survey instrument to measure the factors that constitute the socio-technical context of testing experience. In addition, we state our hypotheses about how these factors impact testing experience and explain the considerations and process that led to the construction of the survey questions.