"That's Not Good Science!": An Argument for the Thoughtful Use of Formative Situations in Research through Design

TL;DR

The paper addresses the bias toward summative evaluation in RtD within HCI and argues for formal recognition of formative evaluations conducted on mid-fidelity prototypes. It introduces formative situations as a frame for evaluating mid-fidelity RtD work and proposes five criteria (C1–C5) to guide rigor and reporting. The authors discuss how such formative work can decouple publication from fully finished products and enable broader knowledge sharing across labs and with industry. They call for community engagement to refine the criteria and to expand acceptance of formative contributions as legitimate scientific knowledge, promoting a more flexible, practice-oriented research culture.

Abstract

Most currently accepted approaches to evaluating Research through Design (RtD) presume that design prototypes are finalized and ready for robust testing in laboratory or in-the-wild settings. However, it is also valuable to assess designs at intermediate phases with mid-fidelity prototypes, not just to inform an ongoing design process, but also to glean knowledge of broader use to the research community. We propose 'formative situations' as a frame for examining mid-fidelity prototypes-in-process in this way. We articulate a set of criteria to help the community better assess the rigor of formative situations, in the service of opening conversation about establishing formative situations as a valuable contribution type within the RtD community.

Figures (1)

• Figure 1: Formative Situations in RtD