Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Input Conditioned Graph Generation for Language Agents

Lukas Vierling, Jie Fu, Kai Chen

TL;DR

This research aims to develop both learnable and dynamic agents by using an existing framework that abstracts language agents as graphs to generate edges that represent the flow of communication within the graph based on the given input, thereby adjusting the internal communication of a language agent.

Abstract

Recent progress in Large Language Models (LLMs) and language agents has demonstrated significant promise for various future applications across multiple disciplines. While traditional approaches to language agents often rely on fixed, handcrafted designs, our research aims to develop both learnable and dynamic agents. Our method uses an existing framework that abstracts language agents as graphs. Within this graph framework, we aim to learn a model that can generate edges for every given input to the language agent. This allows us to generate edges that represent the flow of communication within the graph based on the given input, thereby adjusting the internal communication of a language agent. We learn to generate these edges using a pretrained LLM that is fine-tuned with reinforcement learning. This LLM can be fine-tuned on several datasets simultaneously, and we hypothesize that the model learns to adapt to these different domains during training, achieving good overall performance when encountering data from different domains during deployment. We demonstrate that our approach surpasses the previous static approach by nearly 6% accuracy on a combined dataset of MMLU and CMMLU, and by more than 10% when trained with a sparsity-inducing loss. It also performs superior in additional experiments conducted with the MMLU and Mini Crossword Puzzles datasets. The code is available at https://github.com/lukasVierling/DynamicGPTSwarm.

Input Conditioned Graph Generation for Language Agents

TL;DR

This research aims to develop both learnable and dynamic agents by using an existing framework that abstracts language agents as graphs to generate edges that represent the flow of communication within the graph based on the given input, thereby adjusting the internal communication of a language agent.

Abstract

Recent progress in Large Language Models (LLMs) and language agents has demonstrated significant promise for various future applications across multiple disciplines. While traditional approaches to language agents often rely on fixed, handcrafted designs, our research aims to develop both learnable and dynamic agents. Our method uses an existing framework that abstracts language agents as graphs. Within this graph framework, we aim to learn a model that can generate edges for every given input to the language agent. This allows us to generate edges that represent the flow of communication within the graph based on the given input, thereby adjusting the internal communication of a language agent. We learn to generate these edges using a pretrained LLM that is fine-tuned with reinforcement learning. This LLM can be fine-tuned on several datasets simultaneously, and we hypothesize that the model learns to adapt to these different domains during training, achieving good overall performance when encountering data from different domains during deployment. We demonstrate that our approach surpasses the previous static approach by nearly 6% accuracy on a combined dataset of MMLU and CMMLU, and by more than 10% when trained with a sparsity-inducing loss. It also performs superior in additional experiments conducted with the MMLU and Mini Crossword Puzzles datasets. The code is available at https://github.com/lukasVierling/DynamicGPTSwarm.
Paper Structure (41 sections, 10 equations, 5 figures, 5 tables)

This paper contains 41 sections, 10 equations, 5 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Towards Learned and Dynamic Language Agents
  3. Related Work
  4. Language Agents in Role Play Setting
  5. Dynamic Language Agents
  6. Methodology
  7. Language Agents as Graphs
  8. Static Edge Probabilities
  9. Input-Conditional Edge Probabilities
  10. Design of Our Methodology
  11. Experiments
  12. Crosswords Puzzle Experiment
  13. Adversarial Agent Detection
  14. Specialized Agents Experiment
  15. Edge Reduction on Specialized Agents
  16. ...and 26 more sections

Figures (5)

  • Figure 1: Comparative visualization of edge probabilities in graphs learned by Dynamic Graph and Static Graph. Node 0 is the final decision node, nodes 1 to 8 are truthful agents, and nodes 9 to 16 are adversarial agents. Self-loops and connections from the final decision node to any other node are set to 0.
  • Figure 2: Comparative visualization of edge probabilities on MMLU and CMMLU datasets. Node 0 is the final decision node, nodes 1 to 4 are truthful agents using Gemma-7B-it, and nodes 5 to 8 are truthful agents using BlueLM-7B-chat. Notably, self-loops as well as connections from the final decision node to any other node are not allowed and thereby 0.
  • Figure 3: Comparative visualization of edge probabilities learned by different methods on old and MMLU datasets. Node 0 is the final decision node, nodes 1 to 4 are Gemma-7B-it agents, and nodes 5 to 8 are BlueLM-7B-chat agents. Self-loops and connections from the final decision node to any other node are not allowed.
  • Figure 4: This is a sample graph for the crosswords experiment setup.
  • Figure 5: Probabilities for sampling an edge in the graph by Dynamic Graph with the reduced model. Node 0 is the final decision node, nodes 1 to 4 are truthful agents using Gemma-7B-it, and nodes 5 to 8 are truthful agents using BlueLM-7B-chat. Notably, self-loops as well as connections from the final decision node to any other node are not allowed and thereby 0.