APPL: A Prompt Programming Language for Harmonious Integration of Programs and Large Language Model Prompts

TL;DR

APPL introduces a Prompt Programming Language that bridges conventional programming and LLM prompts, offering Python-native syntax, automatic asynchronous parallelization, and a tracing system for failure recovery. By embedding prompts inside Python functions and enabling tool use through automatic tool specification extraction, APPL simplifies the construction of complex LLM-enabled workflows. The approach yields concise, parallelizable programs across CoT-SC, ReAct-like tool use, and multi-agent chat scenarios, with speedups closely tracking theoretical estimates via a form of Amdahl’s law. This work demonstrates practical, scalable integration of LLM prompts with traditional code, enabling more maintainable and efficient AI-assisted applications across domains requiring tool integration and dynamic prompt assembly.

Abstract

Large Language Models (LLMs) have become increasingly capable of handling diverse tasks with the aid of well-crafted prompts and integration of external tools, but as task complexity rises, the workflow involving LLMs can be complicated and thus challenging to implement and maintain. To address this challenge, we propose APPL, A Prompt Programming Language that acts as a bridge between computer programs and LLMs, allowing seamless embedding of prompts into Python functions, and vice versa. APPL provides an intuitive and Python-native syntax, an efficient parallelized runtime with asynchronous semantics, and a tracing module supporting effective failure diagnosis and replaying without extra costs. We demonstrate that APPL programs are intuitive, concise, and efficient through three representative scenarios: Chain-of-Thought with self-consistency (CoT-SC), ReAct tool use agent, and multi-agent chat. Experiments on three parallelizable workflows further show that APPL can effectively parallelize independent LLM calls, with a significant speedup ratio that almost matches the estimation.

Figures (17)

• Figure 1: We introduce APPL, a prompt programming language that integrates conventional programs and natural language prompts to provide a unified interface for users to access computers and LLMs together.APPL also facilitates users fusing the strengths of computer programs and LLMs by providing convenient conventions between them.
• Figure 2: (a) The APPL program for answering multiple questions about a quotation by first extracting the author's name. In this APPL program, the @ppl marks the function to be APPL function, which provides a context that interactive expression statements (highlighted lines) can be interacted with. Passing the context across APPL functions is implicitly handled along with function calls, mimicking the process of stack frame management. The gen function requests an LLM generation using the prompts stored in the context, with other optional arguments following the OpenAI's API openaiapi. For instance, the gen in line 4 uses the prompt composed by (highlighted) lines 7,8,10 (since context is copied), and 3. The LLM generation is computed asynchronously and only synchronizes when the value is needed, therefore independent calls are automatically parallelized as shown in line 11. (b) A Python program implementing the same function uses more codes, even with the helper functions. Furthermore, extra codes are needed to parallelize get_answer function calls. (c) Example usage for the program and (d) its corresponding LLM prompts and example responses.
• Figure 3: Overview of the APPL runtime. The code is executed following Python control flows, with prompts and LLM responses being captured into the conversation (can be retrieved by convo(), similar to locals() and globals() in the Python frame). The gen uses the conversation captured so far as prompts to call LLMs and is executed asynchronously, so that independent calls can be parallelized easily. APPL allows using Python functions as tools for LLMs by extracting information from their docstrings and signatures. When tools are provided, their specifications are included in the request and the returned tool calls can be easily executed to get results. See detailed code in Figure \ref{['fig:react-appl']}.
• Figure 4: Illustration of the usage of Definitions and Prompt Compositors.
• Figure 5: Illustration of four different ways of inter-function context passing. Usage examples can be found in Section \ref{['sec:multi-agent']} and Appendix \ref{['app:ctx']}. Prompts connected by solid arrows belong to the same prompt context. Dashed arrows represent the control flow of function calls and returns.