Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LikeThis! Empowering App Users to Submit UI Improvement Suggestions Instead of Complaints

Jialiang Wei, Ali Ebrahimi Pourasad, Walid Maalej

TL;DR

This work aims at empowering users to submit better and more constructive UI feedback with concrete suggestions on how to improve the app, and proposes LikeThis!, a GenAI-based approach that takes a user comment with the corresponding screenshot to immediately generate multiple improvement alternatives.

Abstract

User feedback is crucial for the evolution of mobile apps. However, research suggests that users tend to submit uninformative, vague, or destructive feedback. Unlike recent AI4SE approaches that focus on generating code and other development artifacts, our work aims at empowering users to submit better and more constructive UI feedback with concrete suggestions on how to improve the app. We propose LikeThis!, a GenAI-based approach that takes a user comment with the corresponding screenshot to immediately generate multiple improvement alternatives, from which the user can easily choose their preferred option. To evaluate LikeThis!, we first conducted a model benchmarking study based on a public dataset of carefully critiqued UI designs. The results show that GPT-Image-1 significantly outperformed three other state-of-the-art image generation models in improving the designs to address UI issues while keeping the fidelity and without introducing new issues. An intermediate step in LikeThis! is to generate a solution specification before sketching the design as a key to achieving effective improvement. Second, we conducted a user study with 10 production apps, where 15 users used LikeThis! to submit their feedback on encountered issues. Later, the developers of the apps assessed the understandability and actionability of the feedback with and without generated improvements. The results show that our approach helps generate better feedback from both user and developer perspectives, paving the way for AI-assisted user-developer collaboration.

LikeThis! Empowering App Users to Submit UI Improvement Suggestions Instead of Complaints

TL;DR

This work aims at empowering users to submit better and more constructive UI feedback with concrete suggestions on how to improve the app, and proposes LikeThis!, a GenAI-based approach that takes a user comment with the corresponding screenshot to immediately generate multiple improvement alternatives.

Abstract

User feedback is crucial for the evolution of mobile apps. However, research suggests that users tend to submit uninformative, vague, or destructive feedback. Unlike recent AI4SE approaches that focus on generating code and other development artifacts, our work aims at empowering users to submit better and more constructive UI feedback with concrete suggestions on how to improve the app. We propose LikeThis!, a GenAI-based approach that takes a user comment with the corresponding screenshot to immediately generate multiple improvement alternatives, from which the user can easily choose their preferred option. To evaluate LikeThis!, we first conducted a model benchmarking study based on a public dataset of carefully critiqued UI designs. The results show that GPT-Image-1 significantly outperformed three other state-of-the-art image generation models in improving the designs to address UI issues while keeping the fidelity and without introducing new issues. An intermediate step in LikeThis! is to generate a solution specification before sketching the design as a key to achieving effective improvement. Second, we conducted a user study with 10 production apps, where 15 users used LikeThis! to submit their feedback on encountered issues. Later, the developers of the apps assessed the understandability and actionability of the feedback with and without generated improvements. The results show that our approach helps generate better feedback from both user and developer perspectives, paving the way for AI-assisted user-developer collaboration.
Paper Structure (37 sections, 4 figures, 8 tables)

This paper contains 37 sections, 4 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Generating User Feedback with $LikeThis!$
  3. Workflow
  4. Suggestion Generation
  5. UI Editing
  6. iOS Implementation
  7. Models Benchmarking Study
  8. Dataset Preparation
  9. Evaluation Protocol
  10. RQ1.1. Model Performance
  11. Evaluated Models
  12. Evaluation Method
  13. Evaluation Result
  14. RQ1.2. Impact of Masking
  15. Evaluation Settings
  16. ...and 22 more sections

Figures (4)

  • Figure 1: Overview of $LikeThis!$. A user reports an app issue along with a screenshot (1). The AI pipeline generates n solution specifications (2), which are used to create improvement suggestions (3). The user selects their preferred suggestion (4) and submits it along with the issue to developers (5).
  • Figure 2: $LikeThis!$ iOS app enables users to report issues by capturing a screenshot, describing their issue, and optionally marking an area (Screens 1–2). Here, the user reports accidentally tapping the call button while entering a number. $LikeThis!$ generates improvement suggestions (Screens 3–4). Users can then select a suggestion to submit, edit, or reject all. In this case, the user chooses to change the call button to a Slide-To-Call control and submits it with an optional comment (Screen 5).
  • Figure 3: Data sampling from UICrit dataset for RQ1.
  • Figure 4: Evaluation tool to assess generated UIs.