LLM-Driven Cost-Effective Requirements Change Impact Analysis

TL;DR

ProReFiCIA tackles the expensive problem of change impact analysis in requirements engineering by leveraging prompt-engineered large language models in a two-stage pipeline (robust model–prompt selection; refinement and filtering). It demonstrates high recall (93.3%–95.8%) with minimal analyst review (2.1%–8.5% of requirements) across two datasets, using a training-free approach that does not rely on pre-built training data. The study systematically analyzes 64 prompts across five LLMs, identifies the most robust model–prompt pair (LLaMa with P30), and shows that refinement and filtering further boost effectiveness while keeping costs low. Compared with baselines such as NARCIA and RAG-based methods, ProReFiCIA delivers superior performance with substantially lower review effort, highlighting its practical value for scalable CIA in industry contexts.

Abstract

Requirements are inherently subject to changes throughout the software development lifecycle. Within the limited budget available to requirements engineers, manually identifying the impact of such changes on other requirements is both error-prone and effort-intensive. That might lead to overlooked impacted requirements, which, if not properly managed, can cause serious issues in the downstream tasks. Inspired by the growing potential of large language models (LLMs) across diverse domains, we propose ProReFiCIA, an LLM-driven approach for automatically identifying the impacted requirements when changes occur. We conduct an extensive evaluation of ProReFiCIA using several LLMs and prompts variants tailored to this task. Using the best combination of an LLM and a prompt variant, ProReFiCIA achieves a recall of 93.3% on a benchmark dataset and 95.8% on a newly created industry dataset, demonstrating its strong effectiveness in identifying impacted requirements. Further, the cost of applying ProReFiCIA remains small, as the engineer only needs to review the generated results, which represent between 2.1% and 8.5% of the entire set of requirements.

Figures (4)

• Figure 1: Example of three change rationales and their impact on requirements. $C_i$ denotes a change request submitted by the user, and $REQ_i$ represents the $i^{th}$ requirement in the documentation.
• Figure 2: Overview of the steps in the study. Stage 1 focuses on identifying the most effective and generalizable combination of model and prompt. Stage 2 presents our proposed approach, ProReFiCIA, which is designed to automatically generate an impact set for a given change rationale.
• Figure 3: Overview of the development of the prompts. Text 2 (gray circle) in the instruction section is mandatory for the prompt to define the task.
• Figure 4: Box plots depicting the distribution of 64 F2 score results for each LLM across the WASP and SAT-DLink datasets.