AIOS Compiler: LLM as Interpreter for Natural Language Programming and Flow Programming of AI Agents

TL;DR

Natural language programming remains challenging due to vagueness and ambiguity. This paper presents CoRE, a Code Representation and Execution framework that unifies natural language programming, pseudo-code, and flow programming under a single CoRE language and uses an LLM as an interpreter to execute agent programs. It introduces memory-based observation retrieval, structured prompt construction, tool invocation, and branching analysis to guide stepwise execution. Experiments on OpenAGI across GPT-4 and Mixtral show CoRE achieves superior average planning performance compared with baselines. The approach democratizes programming of AI agents and offers a practical pathway to NL-driven agent development, with future work on automated NL instruction generation, multilingual support, and real-time debugging.

Abstract

Since their inception, programming languages have trended towards greater readability and lower barriers for programmers. Following this trend, natural language can be a promising type of programming language that provides great flexibility and usability and helps towards the democracy of programming. However, the inherent vagueness, ambiguity, and verbosity of natural language pose significant challenges in developing an interpreter that can accurately understand the programming logic and execute instructions written in natural language. Fortunately, recent advancements in Large Language Models (LLMs) have demonstrated remarkable proficiency in interpreting complex natural language. Inspired by this, we develop a novel system for Code Representation and Execution (CoRE), which employs LLM as interpreter to interpret and execute natural language instructions. The proposed system unifies natural language programming, pseudo-code programming, and flow programming under the same representation for constructing language agents, while LLM serves as the interpreter to interpret and execute the agent programs. In this paper, we begin with defining the programming syntax that structures natural language instructions logically. During the execution, we incorporate external memory to minimize redundancy. Furthermore, we equip the designed interpreter with the capability to invoke external tools, compensating for the limitations of LLM in specialized domains or when accessing real-time information. This work is open-source at https://github.com/agiresearch/CoRE, https://github.com/agiresearch/OpenAGI, and https://github.com/agiresearch/AIOS.

Figures (6)

• Figure 1: In our CoRE system, we design the CoRE language to unify natural language programming, pseudo-code programming, and flow programming in the same syntax representative. We use the program for OpenAGI openagi platform as an example.
• Figure 2: An example showing how the CoRE system executes one step.
• Figure 3: An overview of the CoRE LLM interpreter system.
• Figure 4: An example showing how the CoRE system retrieves relevant information.
• Figure 5: An example showing how the CoRE system analyze the output from the LLM interpreter.