Promptware Engineering: Software Engineering for LLM Prompt Development

TL;DR

The paper addresses the ad hoc nature of prompt development for LLM-based software by proposing promptware engineering, a framework that adapts traditional software engineering to prompts and LLM runtimes. It defines core concepts, contrasts promptware with conventional software, and presents a detailed roadmap with 24 research opportunities across requirements, design, implementation, testing, debugging, and evolution. Key contributions include characterizing promptware’s unique language and runtime properties, formalizing design patterns, proposing prompt compilation and prompt-centric languages, and outlining systematic testing, debugging, and evolution strategies. The work aims to provide a disciplined, scalable approach to developing reliable, secure, and efficient LLM-based software, with practical impact for practitioners and researchers building prompt-driven systems.

Abstract

Large Language Models (LLMs) are increasingly integrated into software applications, with prompts serving as the primary 'programming' interface to guide their behavior. As a result, a new software paradigm, promptware, has emerged, using natural language prompts to interact with LLMs and enabling complex tasks without traditional coding. Unlike traditional software, which relies on formal programming languages and deterministic runtime environments, promptware is based on ambiguous, unstructured, and context-dependent natural language and operates on LLMs as runtime environments, which are probabilistic and non-deterministic. These fundamental differences introduce unique challenges in prompt development. In practice, prompt development is largely ad hoc and experimental, relying on a time-consuming trial-and-error process - a challenge we term the 'promptware crisis.' To address this, we propose promptware engineering, a new methodology that adapts established software engineering principles to the process of prompt development. Building on decades of success in traditional software engineering, we envision a systematic framework that includes prompt requirements engineering, design, implementation, testing, debugging, and evolution. Unlike traditional software engineering, our framework is specifically tailored to the unique characteristics of prompt development. This paper outlines a comprehensive roadmap for promptware engineering, identifying key research directions and offering actionable insights to advance LLM-based software development.

Figures (2)

• Figure 1: Comparison of the traditional software paradigm and promptware.
• Figure 2: Roadmap for promptware engineering, highlighting key activities alongside associated research opportunities (O1, O2, etc.) and the relevant promptware characteristics (C1 to C10) to be considered for each opportunity.