Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Leveraging LLMs for the Quality Assurance of Software Requirements

Sebastian Lubos, Alexander Felfernig, Thi Ngoc Trang Tran, Damian Garber, Merfat El Mansi, Seda Polat Erdeniz, Viet-Man Le

TL;DR

The paper investigates using a large language model to assess software requirement quality per ISO 29148, enabling classification of quality flaws, transparent explanations, and improvement proposals. Through an empirical study with software engineers on two project datasets (a generated Stopwatch project and a real DigitalHome project), the authors show that LLMs can identify most quality issues and provide plausible explanations, though human review remains essential due to false positives and partial agreement. The work demonstrates the potential of LLM-assisted requirements QA to streamline reviews and boost stakeholder trust, while outlining limitations and directions for scaling to real-world RE with retrieval augmentation and broader evaluation. Overall, the approach offers a promising, explainable, and collaborative path toward higher-quality software requirements.

Abstract

Successful software projects depend on the quality of software requirements. Creating high-quality requirements is a crucial step toward successful software development. Effective support in this area can significantly reduce development costs and enhance the software quality. In this paper, we introduce and assess the capabilities of a Large Language Model (LLM) to evaluate the quality characteristics of software requirements according to the ISO 29148 standard. We aim to further improve the support of stakeholders engaged in requirements engineering (RE). We show how an LLM can assess requirements, explain its decision-making process, and examine its capacity to propose improved versions of requirements. We conduct a study with software engineers to validate our approach. Our findings emphasize the potential of LLMs for improving the quality of software requirements.

Leveraging LLMs for the Quality Assurance of Software Requirements

TL;DR

The paper investigates using a large language model to assess software requirement quality per ISO 29148, enabling classification of quality flaws, transparent explanations, and improvement proposals. Through an empirical study with software engineers on two project datasets (a generated Stopwatch project and a real DigitalHome project), the authors show that LLMs can identify most quality issues and provide plausible explanations, though human review remains essential due to false positives and partial agreement. The work demonstrates the potential of LLM-assisted requirements QA to streamline reviews and boost stakeholder trust, while outlining limitations and directions for scaling to real-world RE with retrieval augmentation and broader evaluation. Overall, the approach offers a promising, explainable, and collaborative path toward higher-quality software requirements.

Abstract

Successful software projects depend on the quality of software requirements. Creating high-quality requirements is a crucial step toward successful software development. Effective support in this area can significantly reduce development costs and enhance the software quality. In this paper, we introduce and assess the capabilities of a Large Language Model (LLM) to evaluate the quality characteristics of software requirements according to the ISO 29148 standard. We aim to further improve the support of stakeholders engaged in requirements engineering (RE). We show how an LLM can assess requirements, explain its decision-making process, and examine its capacity to propose improved versions of requirements. We conduct a study with software engineers to validate our approach. Our findings emphasize the potential of LLMs for improving the quality of software requirements.
Paper Structure (22 sections, 2 equations, 2 figures, 8 tables)

This paper contains 22 sections, 2 equations, 2 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Quality Characteristics of Software Requirements
  4. Instructing LLMs to Evaluate Software Requirements
  5. Related Work
  6. Quality Assurance of Software Requirements with LLMs
  7. Methodology
  8. Study Participants
  9. Dataset and Preprocessing
  10. Stopwatch Project
  11. DigitalHome Project
  12. LLM-based Requirement Evaluation
  13. Experimental Setup
  14. Results
  15. Agreement between Study Participants and LLM (R1)
  16. ...and 7 more sections

Figures (2)

  • Figure 1: LLM prompt template used to evaluate a software requirement for a specified quality characteristic and project. Variables are written in curly brackets "{…}".
  • Figure 2: LLM prompt template used to generate software requirements for an example project to implement a Stopwatch app for Android smartphones. Variables are written in curly brackets "{…}".