Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Multilingual Crowd-Based Requirements Engineering Using Large Language Models

Arthur Pilone, Paulo Meirelles, Fabio Kon, Walid Maalej

TL;DR

An LLM-powered approach called DeeperMatcher that helps agile teams use crowd-based requirements engineering (CrowdRE) in their issue and task management and discusses further refinements needed for reliable crowd-based requirements engineering with multilingual support.

Abstract

A central challenge for ensuring the success of software projects is to assure the convergence of developers' and users' views. While the availability of large amounts of user data from social media, app store reviews, and support channels bears many benefits, it still remains unclear how software development teams can effectively use this data. We present an LLM-powered approach called DeeperMatcher that helps agile teams use crowd-based requirements engineering (CrowdRE) in their issue and task management. We are currently implementing a command-line tool that enables developers to match issues with relevant user reviews. We validated our approach on an existing English dataset from a well-known open-source project. Additionally, to check how well DeeperMatcher works for other languages, we conducted a single-case mechanism experiment alongside developers of a local project that has issues and user feedback in Brazilian Portuguese. Our preliminary analysis indicates that the accuracy of our approach is highly dependent on the text embedding method used. We discuss further refinements needed for reliable crowd-based requirements engineering with multilingual support.

Multilingual Crowd-Based Requirements Engineering Using Large Language Models

TL;DR

An LLM-powered approach called DeeperMatcher that helps agile teams use crowd-based requirements engineering (CrowdRE) in their issue and task management and discusses further refinements needed for reliable crowd-based requirements engineering with multilingual support.

Abstract

A central challenge for ensuring the success of software projects is to assure the convergence of developers' and users' views. While the availability of large amounts of user data from social media, app store reviews, and support channels bears many benefits, it still remains unclear how software development teams can effectively use this data. We present an LLM-powered approach called DeeperMatcher that helps agile teams use crowd-based requirements engineering (CrowdRE) in their issue and task management. We are currently implementing a command-line tool that enables developers to match issues with relevant user reviews. We validated our approach on an existing English dataset from a well-known open-source project. Additionally, to check how well DeeperMatcher works for other languages, we conducted a single-case mechanism experiment alongside developers of a local project that has issues and user feedback in Brazilian Portuguese. Our preliminary analysis indicates that the accuracy of our approach is highly dependent on the text embedding method used. We discuss further refinements needed for reliable crowd-based requirements engineering with multilingual support.
Paper Structure (12 sections, 3 figures)

This paper contains 12 sections, 3 figures.

Table of Contents

  1. Introduction
  2. Approach
  3. Implementation
  4. Preliminary Evaluation
  5. Evaluation Goal
  6. Data Acquisition
  7. Evaluation Methodology
  8. Quantitative Results
  9. Qualitative Results
  10. Discussion
  11. Conclusion
  12. Artifacts Availability

Figures (3)

  • Figure 1: Core components of the proposed architecture. In a darker shade of gray, we highlight the components that receive data from both user reviews and issues. Every component depicted can be switched or adapted for the needs of specific teams or projects.
  • Figure 2: Screenshot of matches identified by DeeperMatcher when prompted with a review from Table III of the DeepMatcher proof of concept haering2021.
  • Figure 3: Suggested issues for a review requesting a new app screen: Although the issue related to the creation of the new screen is not listed, the first suggested match is a fix for a problem with the existing screen.