Supporting Stakeholder Requirements Expression with LLM Revisions: An Empirical Evaluation

TL;DR

This paper tackles articulation barriers in requirements elicitation by evaluating a stakeholder-centered approach where Large Language Models (LLMs) revise and clarify stakeholder statements in real time. In a guided study with 26 IDE users, 130 paired requirement statements were generated and revised using a GDPR-compliant GPT-4o prompt template, with stakeholders validating the revisions. Results show LLM revisions significantly improve alignment with stakeholder intent, readability, reasoning, and unambiguity, while surfacing tacit details and enhancing understanding; only a small fraction introduced errors. The work demonstrates that LLMs can serve as articulation and validation aids in early RE elicitation when kept in the loop with stakeholders, offering design guidance for AI-assisted elicitation tools and highlighting the importance of human oversight to prevent semantic drift.

Abstract

Stakeholders often struggle to accurately express their requirements due to articulation barriers arising from limited domain knowledge or from cognitive constraints. This can cause misalignment between expressed and intended requirements, complicating elicitation and validation. Traditional elicitation techniques, such as interviews and follow-up sessions, are time-consuming and risk distorting stakeholders' original intent across iterations. Large Language Models (LLMs) can infer user intentions from context, suggesting potential for assisting stakeholders in expressing their needs. This raises the questions of (i) how effectively LLMs can support requirement expression and (ii) whether such support benefits stakeholders with limited domain expertise. We conducted a study with 26 participants who produced 130 requirement statements. Each participant first expressed requirements unaided, then evaluated LLM-generated revisions tailored to their context. Participants rated LLM revisions significantly higher than their original statements across all dimensions-alignment with intent, readability, reasoning, and unambiguity. Qualitative feedback further showed that LLM revisions often surfaced tacit details stakeholders considered important and helped them better understand their own requirements. We present and evaluate a stakeholder-centered approach that leverages LLMs as articulation aids in requirements elicitation and validation. Our results show that LLM-assisted reformulation improves perceived completeness, clarity, and alignment of requirements. By keeping stakeholders in the validation loop, this approach promotes responsible and trustworthy use of AI in Requirements Engineering.

Figures (4)

• Figure 1: Methodology of our study design. Stakeholder and generated artifacts are colored green, LLM and generated artifacts are colored blue.
• Figure 2: Participant ratings comparing LLM-revised to original requirements in four dimensions. Bars show the proportion of ratings on a five-point scale from LLM-revisions being “much worse” to “much better.”
• Figure 3: Proportion of perceived improvements or issues in LLM-revised requirements ($N = 130$). Each bar shows the percentage of participants in the low- and high-experience groups answering “Yes” or “No.”
• Figure 4: Comparison of responses from low- and high-experience stakeholders.