Goal2Story: A Multi-Agent Fleet based on Privately Enabled sLLMs for Impacting Mapping on Requirements Elicitation

TL;DR

Goal2Story introduces a private-sLLM driven multi-agent fleet that uses the Impact Mapping framework to automate goal-driven requirements elicitation and generate user stories. It pairs this with StorySeek, a semi-automatic dataset of 1,005 IM-Result and Project-Info records, and two evaluation metrics, FHR and QuACE, to assess factual alignment and quality. Empirical results show Goal2Story outperforms a Super-Agent baseline across FHR and QuACE, with CoT prompting and agent profiling providing additional gains and enabling discovery of latent needs. The work demonstrates that privacy-preserving, cost-efficient LLM configurations can effectively support RE automation, offering practical benefits for agile teams under data-security constraints.

Abstract

As requirements drift with rapid iterations, agile development becomes the dominant paradigm. Goal-driven Requirements Elicitation (RE) is a pivotal yet challenging task in agile project development due to its heavy tangling with adaptive planning and efficient collaboration. Recently, AI agents have shown promising ability in supporting requirements analysis by saving significant time and effort for stakeholders. However, current research mainly focuses on functional RE, and research works have not been reported bridging the long journey from goal to user stories. Moreover, considering the cost of LLM facilities and the need for data and idea protection, privately hosted small-sized LLM should be further utilized in RE. To address these challenges, we propose Goal2Story, a multi-agent fleet that adopts the Impact Mapping (IM) framework while merely using cost-effective sLLMs for goal-driven RE. Moreover, we introduce a StorySeek dataset that contains over 1,000 user stories (USs) with corresponding goals and project context information, as well as the semi-automatic dataset construction method. For evaluation, we proposed two metrics: Factuality Hit Rate (FHR) to measure consistency between the generated USs with the dataset and Quality And Consistency Evaluation (QuACE) to evaluate the quality of the generated USs. Experimental results demonstrate that Goal2Story outperforms the baseline performance of the Super-Agent adopting powerful LLMs, while also showcasing the performance improvements in key metrics brought by CoT and Agent Profile to Goal2Story, as well as its exploration in identifying latent needs.

Figures (5)

• Figure 1: Inviting Goal2Story: A Goal-Driven way to mitigate the suffering of practicing requirements elicitation adopting sLLMs, with impact mapping framework
• Figure 2: How Goal2Story Functions. The left section presents the StorySeek dataset along with the semi-automatic dataset construction method. The upper right section outlines the basic architecture and workflow of Goal2Story, incorporating CoT and Profiling features. The lower right section provides the evaluation results based on FHR and QuACE metrics, as well as insights into latent needs identification.
• Figure 3: Workflow of the Goal2Story
• Figure 4: Performance of Goal2Story and Super-Agent (with CoT) with different models on FHR and QuACE
• Figure 5: Performance of Goal2Story with different features (CoT and Profile) on FHR