Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Solution-oriented Agent-based Models Generation with Verifier-assisted Iterative In-context Learning

Tong Niu, Weihao Zhang, Rong Zhao

TL;DR

This work tackles the challenge of automatically generating executable and verifiable ABMs and their problem-solving policies for complex systems. It introduces SAGE, a verifier-assisted iterative in-context learning framework with a Modeling stage that builds executable ABMs from semi-structured representations and a Solving stage that optimizes solutions via verification-driven CoT prompting. The approach employs a two-level verifier to ensure executability and objective satisfaction, and it is validated on a solution-oriented ABMs dataset across multiple domains, showing significant gains over No-SAGE baselines. By reducing the need for extensive domain-specific training and enabling cross-domain ABM generation and evaluation, SAGE broadens access to rapid ABM development and policy experimentation in complex systems research.

Abstract

Agent-based models (ABMs) stand as an essential paradigm for proposing and validating hypothetical solutions or policies aimed at addressing challenges posed by complex systems and achieving various objectives. This process demands labor-intensive endeavors and multidisciplinary expertise. Large language models (LLMs) encapsulating cross-domain knowledge and programming proficiency could potentially alleviate the difficulty of this process. However, LLMs excel in handling sequential information, making it challenging for analyzing the intricate interactions and nonlinear dynamics inherent in ABMs. Additionally, due to the lack of self-evaluation capability of LLMs, relying solely on LLMs is insufficient to effectively accomplish this process. In this paper, we present SAGE, a general solution-oriented ABM generation framework designed for automatic modeling and generating solutions for targeted problems. Unlike approaches reliant on expert handcrafting or resource-intensive neural network training, SAGE establishes a verifier-assisted iterative in-context learning process employing large language models (LLMs) to leverages their inherent cross-domain knowledge for tackling intricate demands from diverse domain scenarios. In SAGE, we introduce an semi-structured conceptual representation expliciting the intricate structures of ABMs and an objective representation to guide LLMs in modeling scenarios and proposing hypothetical solutions through in-context learning. To ensure the model executability and solution feasibility, SAGE devises a two-level verifier with chain-of-thought prompting tailored to the complex interactions and non-linear dynamics of ABMs, driving the iterative generation optimization. Moreover, we construct an evaluation dataset of solution-oriented ABMs from open sources.It contains practical models across various domains.

Solution-oriented Agent-based Models Generation with Verifier-assisted Iterative In-context Learning

TL;DR

This work tackles the challenge of automatically generating executable and verifiable ABMs and their problem-solving policies for complex systems. It introduces SAGE, a verifier-assisted iterative in-context learning framework with a Modeling stage that builds executable ABMs from semi-structured representations and a Solving stage that optimizes solutions via verification-driven CoT prompting. The approach employs a two-level verifier to ensure executability and objective satisfaction, and it is validated on a solution-oriented ABMs dataset across multiple domains, showing significant gains over No-SAGE baselines. By reducing the need for extensive domain-specific training and enabling cross-domain ABM generation and evaluation, SAGE broadens access to rapid ABM development and policy experimentation in complex systems research.

Abstract

Agent-based models (ABMs) stand as an essential paradigm for proposing and validating hypothetical solutions or policies aimed at addressing challenges posed by complex systems and achieving various objectives. This process demands labor-intensive endeavors and multidisciplinary expertise. Large language models (LLMs) encapsulating cross-domain knowledge and programming proficiency could potentially alleviate the difficulty of this process. However, LLMs excel in handling sequential information, making it challenging for analyzing the intricate interactions and nonlinear dynamics inherent in ABMs. Additionally, due to the lack of self-evaluation capability of LLMs, relying solely on LLMs is insufficient to effectively accomplish this process. In this paper, we present SAGE, a general solution-oriented ABM generation framework designed for automatic modeling and generating solutions for targeted problems. Unlike approaches reliant on expert handcrafting or resource-intensive neural network training, SAGE establishes a verifier-assisted iterative in-context learning process employing large language models (LLMs) to leverages their inherent cross-domain knowledge for tackling intricate demands from diverse domain scenarios. In SAGE, we introduce an semi-structured conceptual representation expliciting the intricate structures of ABMs and an objective representation to guide LLMs in modeling scenarios and proposing hypothetical solutions through in-context learning. To ensure the model executability and solution feasibility, SAGE devises a two-level verifier with chain-of-thought prompting tailored to the complex interactions and non-linear dynamics of ABMs, driving the iterative generation optimization. Moreover, we construct an evaluation dataset of solution-oriented ABMs from open sources.It contains practical models across various domains.
Paper Structure (13 sections, 7 figures, 3 tables, 2 algorithms)

This paper contains 13 sections, 7 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Automatic Design and Generation of ABM
  4. LLMs for Code generation
  5. In-context Learning
  6. SAGE
  7. "Modeling" Stage
  8. "Solving" Stage
  9. Dataset and Evaluation
  10. Solution-oriented ABMs Dataset
  11. "Modeling" Evaluation
  12. "Solving" Evaluation
  13. Conclusions

Figures (7)

  • Figure 1: The workflow of SAGE.
  • Figure 2: Conceptual representation for ABM description and the generated ABM program.
  • Figure 3: The rectification prompts generated by verifier-level1 and the rectified ABM program.
  • Figure 4: Objective representation for desired solution effects and the generated verification program.
  • Figure 5: The CoT workflow and the generated solutions with enhanced ABM program.
  • ...and 2 more figures