Towards Conversational Development Environments: Using Theory-of-Mind and Multi-Agent Architectures for Requirements Refinement

TL;DR

This work tackles the challenge that foundation models struggle to accurately capture stakeholder requirements during the requirements refinement phase of software development. It introduces AlignMind, a FM-powered multi-agent system augmented with Theory-of-Mind capabilities to iteratively clarify stakeholders' beliefs, desires, and intentions, translating them into refined requirements and an actionable natural-language workflow. Across 150 synthetic use cases, AlignMind outperforms a baseline prompting approach on multiple evaluation rubrics, increases requirement richness, and sustains extended multi-round dialogues, though at a higher token and API-call cost. The results support the vision of intent-first development environments where AI collaborates with software makers to produce requirements that truly reflect stakeholder needs and facilitate robust implementation, while outlining future work in multi-modal integration and domain-specific ToM extensions.

Abstract

Foundation Models (FMs) have shown remarkable capabilities in various natural language tasks. However, their ability to accurately capture stakeholder requirements remains a significant challenge for using FMs for software development. This paper introduces a novel approach that leverages an FM-powered multi-agent system called AlignMind to address this issue. By having a cognitive architecture that enhances FMs with Theory-of-Mind capabilities, our approach considers the mental states and perspectives of software makers. This allows our solution to iteratively clarify the beliefs, desires, and intentions of stakeholders, translating these into a set of refined requirements and a corresponding actionable natural language workflow in the often-overlooked requirements refinement phase of software engineering, which is crucial after initial elicitation. Through a multifaceted evaluation covering 150 diverse use cases, we demonstrate that our approach can accurately capture the intents and requirements of stakeholders, articulating them as both specifications and a step-by-step plan of action. Our findings suggest that the potential for significant improvements in the software development process justifies these investments. Our work lays the groundwork for future innovation in building intent-first development environments, where software makers can seamlessly collaborate with AIs to create software that truly meets their needs.

Figures (12)

• Figure 1: The Terminal-based User Interface (TUI) for AlignMind features a horizontal panel where users can input their username and choose configuration options. To the right, a chat box displays the conversation history with AlignMind, while a text field below allows users to type messages. On the left side, AlignMind displays the generated requirements and workflow.
• Figure 2: An overview of the proposed requirements refinement system, AlignMind, which consists of Router, Requirement Refiner, Workflow Generator, Workflow Refiner, and ToM helper agents.
• Figure 3: An overview of the topic and question generation workflow.
• Figure 4: An overview of the evaluation of AlignMind.
• Figure 5: An overview of the output quality evaluation by a panel of FM judges (RQ1.1).