AutoGRAMS: Autonomous Graphical Agent Modeling Software

TL;DR

AutoGRAMS introduces a graph-based programming language for AI agents that unifies LM prompts, traditional code, and memory into executable autograms. It provides a compiler to translate Python code into AutoGRAMS graphs, an interpreter to run these graphs, and mechanisms for memory, variable scoping, and self-modification, including meta-autograms that design new autograms. The framework emphasizes interpretability, controllability, and safety in multi-step LM interactions, with open-source availability to foster experimentation and extension. By enabling modular subgraphs, function calls, and self-modifying capabilities, AutoGRAMS offers a flexible pathway toward sophisticated, self-improving AI agents and complex conversational systems.

Abstract

We introduce the AutoGRAMS framework for programming multi-step interactions with language models. AutoGRAMS represents AI agents as a graph, where each node can execute either a language modeling instruction or traditional code. Likewise, transitions in the graph can be governed by either language modeling decisions or traditional branch logic. AutoGRAMS supports using variables as memory and allows nodes to call other AutoGRAMS graphs as functions. We show how AutoGRAMS can be used to design highly sophisticated agents, including self-referential agents that can modify their own graph. AutoGRAMS's graph-centric approach aids interpretability, controllability, and safety during the design, development, and deployment of AI agents. We provide our framework as open source at https://github.com/autograms/autograms .

Figures (12)

• Figure 1: Graphical representation of a chatbot created using an autogram with only chat-type nodes. Nodes represents a point where the language model will give a response, and each edge represents a different type of user response. This particular chatbot models an AI tutor that repeatedly quizzes the user about a subject. The chatbot starts with the ask_question node where it asks the user a question, and then decides which node to visit next depending on whether the user's answer is right or wrong. If the answer is wrong, the interaction goes to the answer_wrong node where the chatbot lets the user know the answer is wrong and asks them to try again. If the user answers wrong twice, the chatbot gives the answer. Eventually the chatbot reaches a node where it repeatedly answers questions the user has about the original question, and when the user is done asking questions, the chatbot revisits the original ask_question node to ask another question. Each node executes a specific planned step in the conversation. Each node has a separate instruction for how the model should reply. Each node with multiple outgoing edges contains a multiple choice question that the language model uses to determine which node to visit next. So for instance, in this example, the ask_question node has a transition question of "is the user's answer correct? A. Yes B. No". If after viewing the user's reply, the language model predicts the answer is Yes, then the autogram will go to the answer_right node to get the next instruction. Otherwise it will go to the answer_wrong node.
• Figure 2: Depiction of how a node's instruction is executed in AutoGRAMS. Behavior can vary greatly depending on the type of node. '$' variables are covered in Section \ref{['sec:variables']}
• Figure 3: Depiction of how the next node is selected from a previous one in AutoGRAMS. If the previous instruction executed a function call, the typical transition behavior is overridden and the autogram transitions to the node called in the previous instruction. If the number of defined transitions for that node is more than one, then the nodes transition question and transition choices are used to construct a multiple choice question, which is passed to a language model. The language model's prediction determines which transition to select. If the selected transition is a variable, this is set during transition post-processing. If the selected transition corresponds to the name of an existing node (standard transition), than that node is selected. If the transition is a "return" transition, the previous function calling node is selected as the next node. Lastly, if the transition is a wildcard transition (ending in ".*"), if/else-if/else logic is used to determine which of 2 or more nodes with the same prefix as the transition is selected.
• Figure 4: Example of a set of nodes that use variables. The exact behavior of variable assignment can depend on the type of node, but for thought and chat style nodes, this is the text generated by the language model, and for python-type nodes this is the result of the code execution using the Python interpreter. Variable assignment allows for memories to be stored that can be referenced in future nodes. Variables can be referenced in 2 ways--directly or using $-syntax. Direct use means using a variable as is, and can only be used in fields that will be passed to directly the Python interpreter. This includes the instruction of python-type nodes (see the $python\_function$ node above), and the $boolean\_condition$ attribute of a node that is often used in combination with wild card transitions. $-syntax means embed the variable as a string on the fly; for instance in the example above, the model will reply with the exact text set in the topics variable inserted into the instruction (a $chat\_exact$ node replies with the exact instruction without calling the language model). $-variable references are most straightforwardly used in $instruction$, $transition\_question$ to facilitate on-the-fly instructions to a language model or on-the-fly transition questions that incorporate a memory. With care, $-variable references can also be used in the instruction of python-type nodes to create dynamically changing code, or as elements of the transition list to allow for dynamically changing transitions in the graph.
• Figure 5: Depiction of function calls in AutoGRAMS. (a) depicts a function with chat-style nodes that allows the conversation to return to the calling node when a function returns. (b) depicts a function with thought-style nodes, which is useful for reusing modules and controlling scopes. AutoGRAMS functions can be called from within an autogram or from Python using the autogram.apply_fn() method.