Effort and Size Estimation in Software Projects with Large Language Model-based Intelligent Interfaces
Claudionor N. Coelho, Hanchen Xiong, Tushar Karayil, Sree Koratala, Rex Shang, Jacob Bollinger, Mohamed Shabar, Syam Nair
TL;DR
The paper addresses the challenge of estimating development effort for software systems that employ large language models as intelligent interfaces. It argues that traditional UI/UX-based estimation fails under NL-driven interactions and proposes a planner-based decomposition approach within an AI Agent framework to enumerate necessary data sources, algorithms, and interfaces by generating and evaluating related NL questions. By iterating question generation and subtasks, the method aims to recover the estimation precision once offered by explicit use cases, even in the face of ambiguous NL specifications. The practical impact lies in enabling more reliable planning for LLM-enabled software projects, with explicit scope documentation and non-accessed components identified upfront.
Abstract
The advancement of Large Language Models (LLM) has also resulted in an equivalent proliferation in its applications. Software design, being one, has gained tremendous benefits in using LLMs as an interface component that extends fixed user stories. However, inclusion of LLM-based AI agents in software design often poses unexpected challenges, especially in the estimation of development efforts. Through the example of UI-based user stories, we provide a comparison against traditional methods and propose a new way to enhance specifications of natural language-based questions that allows for the estimation of development effort by taking into account data sources, interfaces and algorithms.