Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Interlinking User Stories and GUI Prototyping: A Semi-Automatic LLM-based Approach

Kristian Kolthoff, Felix Kretzer, Christian Bartelt, Alexander Maedche, Simone Paolo Ponzetto

TL;DR

The paper tackles the challenge of validating natural-language user stories against GUI prototypes and providing actionable implementation recommendations to support rapid GUI prototyping. It introduces a semi-automatic, LLM-driven pipeline that (i) abstracts GUI prototypes into a functional textual representation, (ii) detects whether user stories are implemented, (iii) maps user stories to GUI components, and (iv) generates HTML/CSS-based recommendations. Preliminary evaluation on Rico-derived data shows promising accuracy for implementation detection and a moderate ability to match relevant GUI components, with recommendations demonstrated in HTML/CSS form. The work highlights the potential to reduce prototyping effort and improve requirements elicitation and validation, while outlining limitations and a concrete plan for expanding datasets and user studies to assess practical impact. Overall, the approach provides a feasible pathway toward integrated, AI-assisted GUI prototyping workflows embedded within design tools.

Abstract

Interactive systems are omnipresent today and the need to create graphical user interfaces (GUIs) is just as ubiquitous. For the elicitation and validation of requirements, GUI prototyping is a well-known and effective technique, typically employed after gathering initial user requirements represented in natural language (NL) (e.g., in the form of user stories). Unfortunately, GUI prototyping often requires extensive resources, resulting in a costly and time-consuming process. Despite various easy-to-use prototyping tools in practice, there is often a lack of adequate resources for developing GUI prototypes based on given user requirements. In this work, we present a novel Large Language Model (LLM)-based approach providing assistance for validating the implementation of functional NL-based requirements in a GUI prototype embedded in a prototyping tool. In particular, our approach aims to detect functional user stories that are not implemented in a GUI prototype and provides recommendations for suitable GUI components directly implementing the requirements. We collected requirements for existing GUIs in the form of user stories and evaluated our proposed validation and recommendation approach with this dataset. The obtained results are promising for user story validation and we demonstrate feasibility for the GUI component recommendations.

Interlinking User Stories and GUI Prototyping: A Semi-Automatic LLM-based Approach

TL;DR

The paper tackles the challenge of validating natural-language user stories against GUI prototypes and providing actionable implementation recommendations to support rapid GUI prototyping. It introduces a semi-automatic, LLM-driven pipeline that (i) abstracts GUI prototypes into a functional textual representation, (ii) detects whether user stories are implemented, (iii) maps user stories to GUI components, and (iv) generates HTML/CSS-based recommendations. Preliminary evaluation on Rico-derived data shows promising accuracy for implementation detection and a moderate ability to match relevant GUI components, with recommendations demonstrated in HTML/CSS form. The work highlights the potential to reduce prototyping effort and improve requirements elicitation and validation, while outlining limitations and a concrete plan for expanding datasets and user studies to assess practical impact. Overall, the approach provides a feasible pathway toward integrated, AI-assisted GUI prototyping workflows embedded within design tools.

Abstract

Interactive systems are omnipresent today and the need to create graphical user interfaces (GUIs) is just as ubiquitous. For the elicitation and validation of requirements, GUI prototyping is a well-known and effective technique, typically employed after gathering initial user requirements represented in natural language (NL) (e.g., in the form of user stories). Unfortunately, GUI prototyping often requires extensive resources, resulting in a costly and time-consuming process. Despite various easy-to-use prototyping tools in practice, there is often a lack of adequate resources for developing GUI prototypes based on given user requirements. In this work, we present a novel Large Language Model (LLM)-based approach providing assistance for validating the implementation of functional NL-based requirements in a GUI prototype embedded in a prototyping tool. In particular, our approach aims to detect functional user stories that are not implemented in a GUI prototype and provides recommendations for suitable GUI components directly implementing the requirements. We collected requirements for existing GUIs in the form of user stories and evaluated our proposed validation and recommendation approach with this dataset. The obtained results are promising for user story validation and we demonstrate feasibility for the GUI component recommendations.
Paper Structure (18 sections, 4 figures, 2 tables)

This paper contains 18 sections, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Approach
  4. GUI Prototype Abstraction
  5. User Story Implementation Detection
  6. User Story GUI Component Matching
  7. User Story Implementation Recommendation
  8. Proposed Integrated System
  9. Evaluation
  10. Data Collection
  11. RQ$_{1}$: User Story Implementation Detection
  12. RQ$_{2}$: User Story GUI Component Matching
  13. Results and Discussion
  14. RQ$_{1}$: User Story Implementation Detection
  15. RQ$_{2}$: User Story GUI Component Matching
  16. ...and 3 more sections

Figures (4)

  • Figure 1: Overview of our proposed approach pipeline (A-D)
  • Figure 2: Proposed system integrating user story based validation and GUI recommendations into dedicated prototyping tools
  • Figure 3: GUI component matches to user stories. Redrawn from Rico RICO_Deka_2017 using components from Material Designgoogle_llc_material_2024
  • Figure 4: GUI feature recommendations (randomly sampled)