StudyAlign: A Software System for Conducting Web-Based User Studies with Functional Interactive Prototypes

TL;DR

StudyAlign addresses the complexity of conducting web-based user studies with functional prototypes by delivering a modular, open-source platform that ties together a Backend, Control Panel, Study Frontend, and a Library for prototype logging. It enables end-to-end experiment design, logging, counterbalancing, and study sharing, while supporting external questionnaire tools and cross-platform portability. The authors validate the approach through multiple case studies, discuss benefits for research teams, and outline a path toward broader adoption and community-driven development. The work offers a practical, scalable solution that reduces development effort, enhances replication, and fosters transparency in online HCI research.

Abstract

Interactive systems are commonly prototyped as web applications. This approach enables studies with functional prototypes on a large scale. However, setting up these studies can be complex due to implementing experiment procedures, integrating questionnaires, and data logging. To enable such user studies, we developed the software system StudyAlign which offers: 1) a frontend for participants, 2) an admin panel to manage studies, 3) the possibility to integrate questionnaires, 4) a JavaScript library to integrate data logging into prototypes, and 5) a backend server for persisting log data, and serving logical functions via an API to the different parts of the system. With our system, researchers can set up web-based experiments and focus on the design and development of interactions and prototypes. Furthermore, our systematic approach facilitates the replication of studies and reduces the required effort to execute web-based user studies. We conclude with reflections on using StudyAlign for conducting HCI studies online.

Figures (9)

• Figure 1: This figure visualizes the process of researchers for setting up web-based experiments with StudyAlign. The first two blocks in this process cover sketching the study design, designing and implementing prototypes, creating questionnaires in tools such as Qualtrics, and preparing text documents, such as task briefings. The latter three blocks are then supported by StudyAlign. Researchers use the Control Panel for creating the study and procedure. Instructions support the researchers integrating questionnaires into the procedure, and the Library allows researchers to enable interaction logging in the prototypes. Finally, researchers conduct the experiment by sharing a study link with participants that displays a Study Frontend to guide them through the experiment.
• Figure 2: Overview of the software system StudyAlign. The main components are the Backend, Control Panel, Study Frontend, and Library. The Backend is the core of the system since it holds the logic and persists data in a database. The Control Panel is used by researchers to set up and manage studies. The Study Frontend controls the experiment procedure (see \ref{['fig:navigator']} about its Navigator for details) and is served to the participants' devices. It embeds prototypes and questionnaires and displays them to participants. Questionnaires use a callback function to work with the API. The Library is the counterpart to the Backend's API and is used in the prototype for interaction logging and in the Study Frontend for controlling the procedure.
• Figure 3: A class diagram of the Procedure Config used in the Backend to generate procedures. When a researcher sets up a procedure through the Control Panel, a Procedure Config is created. This Procedure Config has multiple Procedure Config Steps. Each Procedure Config Step has an order number and a flag for counterbalancing. Furthermore, it is assigned a concrete procedure element, such as a Condition, Questionnaire, Text, Pause, or Block. A Block extends Procedure Config and can have multiple Procedure Config Steps. This way, a block can combine multiple elements into one element.
• Figure 4: The Navigator is the conceptual component of StudyAlign which controls the procedure. When a participant views a questionnaire or prototype, the Navigator decides whether the participant is allowed to proceed. The diagram is read from left to right and depicts the protocol. The Navigator connects at the beginning of a questionnaire or prototype, tries to reconnect if the connection fails, and waits for a signal when the task is finished for the participant to continue.
• Figure 5: Screenshot depicting the actual UI of the Study Frontend. In this screenshot, a task briefing that was implemented with a text page element is displayed. The text page displays a title and text instructions. The Navigator UI at the bottom of the screenshot allows participants to go ahead with the procedure by clicking on the "next" button in the bottom right.