Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

AI based Multiagent Approach for Requirements Elicitation and Analysis

Malik Abdul Sami, Muhammad Waseem, Zheying Zhang, Zeeshan Rasheed, Kari Systä, Pekka Abrahamsson

TL;DR

Results corroborate the effectiveness of LLMs in improving and streamlining RE phases by analyzing the semantic similarity and API performance of different models, as well as their effectiveness and efficiency in requirements analysis.

Abstract

Requirements Engineering (RE) plays a pivotal role in software development, encompassing tasks such as requirements elicitation, analysis, specification, and change management. Despite its critical importance, RE faces challenges including communication complexities, early-stage uncertainties, and accurate resource estimation. This study empirically investigates the effectiveness of utilizing Large Language Models (LLMs) to automate requirements analysis tasks. We implemented a multi-agent system that deploys AI models as agents to generate user stories from initial requirements, assess and improve their quality, and prioritize them using a selected technique. In our implementation, we deployed four models, namely GPT-3.5, GPT-4 Omni, LLaMA3-70, and Mixtral-8B, and conducted experiments to analyze requirements on four real-world projects. We evaluated the results by analyzing the semantic similarity and API performance of different models, as well as their effectiveness and efficiency in requirements analysis, gathering users' feedback on their experiences. Preliminary results indicate notable variations in task completion among the models. Mixtral-8B provided the quickest responses, while GPT-3.5 performed exceptionally well when processing complex user stories with a higher similarity score, demonstrating its capability in deriving accurate user stories from project descriptions. Feedback and suggestions from the four project members further corroborate the effectiveness of LLMs in improving and streamlining RE phases.

AI based Multiagent Approach for Requirements Elicitation and Analysis

TL;DR

Results corroborate the effectiveness of LLMs in improving and streamlining RE phases by analyzing the semantic similarity and API performance of different models, as well as their effectiveness and efficiency in requirements analysis.

Abstract

Requirements Engineering (RE) plays a pivotal role in software development, encompassing tasks such as requirements elicitation, analysis, specification, and change management. Despite its critical importance, RE faces challenges including communication complexities, early-stage uncertainties, and accurate resource estimation. This study empirically investigates the effectiveness of utilizing Large Language Models (LLMs) to automate requirements analysis tasks. We implemented a multi-agent system that deploys AI models as agents to generate user stories from initial requirements, assess and improve their quality, and prioritize them using a selected technique. In our implementation, we deployed four models, namely GPT-3.5, GPT-4 Omni, LLaMA3-70, and Mixtral-8B, and conducted experiments to analyze requirements on four real-world projects. We evaluated the results by analyzing the semantic similarity and API performance of different models, as well as their effectiveness and efficiency in requirements analysis, gathering users' feedback on their experiences. Preliminary results indicate notable variations in task completion among the models. Mixtral-8B provided the quickest responses, while GPT-3.5 performed exceptionally well when processing complex user stories with a higher similarity score, demonstrating its capability in deriving accurate user stories from project descriptions. Feedback and suggestions from the four project members further corroborate the effectiveness of LLMs in improving and streamlining RE phases.
Paper Structure (16 sections, 5 figures, 2 tables)

This paper contains 16 sections, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Generative AI in Software Engineering
  3. Proposing Agents for Requirements Analysis
  4. User Story Generation and Quality Assessment Agent
  5. Multiagent Agents for Prioritization
  6. Experimental Design
  7. Experiment design
  8. Example projects
  9. Evaluation and Feedback
  10. Results
  11. Prioritization Results
  12. Discussion
  13. AI agents assistance in requirements analysis (RQ1)
  14. Effectiveness of Different Language Models in Performing Requirements Analysis Tasks (RQ2)
  15. Conclusion
  16. ...and 1 more sections

Figures (5)

  • Figure 1: Process Model of LLM Agents for User Story Generation, check the quality of the story, and Prioritization.
  • Figure 2: Architecture of a Multi-Agent System
  • Figure 3: Metrics Comparison for Four Projects Using Different Models.
  • Figure 4: Prioritization results of Project 1.
  • Figure 5: Prioritization result of Project 2.