Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Leveraging LLM Agents for Automated Optimization Modeling for SASP Problems: A Graph-RAG based Approach

Tianpeng Pan, Wenqiang Pu, Licheng Zhao, Rui Zhou

TL;DR

The paper tackles automated optimization modeling for SASP problems by addressing domain-knowledge gaps in LLM-powered solutions. It introduces MAG-RAG, a framework that combines a multi-agent AOM workflow with a Graph-RAG knowledge graph to guide modeling using domain-specific content, with a retrieval of top-$k$ knowledge where $k=3$. Evaluated on ten classical SASP problems, MAG-RAG outperforms two baselines (Pure MA and Pure LLM) across multiple criteria, as judged by SASP experts, demonstrating improved completeness and correctness. The work highlights practical impact in delivering domain-aware optimization suggestions via structured knowledge graphs, while noting limitations in capturing implicit algorithmic relations and clustering strategies for retrieval, which point to future refinements.

Abstract

Automated optimization modeling (AOM) has evoked considerable interest with the rapid evolution of large language models (LLMs). Existing approaches predominantly rely on prompt engineering, utilizing meticulously designed expert response chains or structured guidance. However, prompt-based techniques have failed to perform well in the sensor array signal processing (SASP) area due the lack of specific domain knowledge. To address this issue, we propose an automated modeling approach based on retrieval-augmented generation (RAG) technique, which consists of two principal components: a multi-agent (MA) structure and a graph-based RAG (Graph-RAG) process. The MA structure is tailored for the architectural AOM process, with each agent being designed based on principles of human modeling procedure. The Graph-RAG process serves to match user query with specific SASP modeling knowledge, thereby enhancing the modeling result. Results on ten classical signal processing problems demonstrate that the proposed approach (termed as MAG-RAG) outperforms several AOM benchmarks.

Leveraging LLM Agents for Automated Optimization Modeling for SASP Problems: A Graph-RAG based Approach

TL;DR

The paper tackles automated optimization modeling for SASP problems by addressing domain-knowledge gaps in LLM-powered solutions. It introduces MAG-RAG, a framework that combines a multi-agent AOM workflow with a Graph-RAG knowledge graph to guide modeling using domain-specific content, with a retrieval of top- knowledge where . Evaluated on ten classical SASP problems, MAG-RAG outperforms two baselines (Pure MA and Pure LLM) across multiple criteria, as judged by SASP experts, demonstrating improved completeness and correctness. The work highlights practical impact in delivering domain-aware optimization suggestions via structured knowledge graphs, while noting limitations in capturing implicit algorithmic relations and clustering strategies for retrieval, which point to future refinements.

Abstract

Automated optimization modeling (AOM) has evoked considerable interest with the rapid evolution of large language models (LLMs). Existing approaches predominantly rely on prompt engineering, utilizing meticulously designed expert response chains or structured guidance. However, prompt-based techniques have failed to perform well in the sensor array signal processing (SASP) area due the lack of specific domain knowledge. To address this issue, we propose an automated modeling approach based on retrieval-augmented generation (RAG) technique, which consists of two principal components: a multi-agent (MA) structure and a graph-based RAG (Graph-RAG) process. The MA structure is tailored for the architectural AOM process, with each agent being designed based on principles of human modeling procedure. The Graph-RAG process serves to match user query with specific SASP modeling knowledge, thereby enhancing the modeling result. Results on ten classical signal processing problems demonstrate that the proposed approach (termed as MAG-RAG) outperforms several AOM benchmarks.

Paper Structure

This paper contains 9 sections, 7 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. The Proposed AOM Approach
  3. The MAG-RAG Pipeline
  4. Graph-RAG Dataset Construction
  5. Knowledge Searching Using Graph-RAG
  6. Experiments
  7. Experimental Setup
  8. Performance Evaluation
  9. Conclusion

Figures (4)

  • Figure 1: The overall workflow of the proposed method.
  • Figure 2: Human optimization modeling procedure for SASP problems.
  • Figure 3: An example of the output generated by Example Extraction Agent
  • Figure 4: Statistical results on overall scores. (A) Frequency of different methods achieving the highest scores across various metrics. (B) Frequency of scoring gains (positive, negative and no gain) obtained with prior knowledge (both pure MA and MAG-RAG) compared to pure LLM. (C) Frequency of scoring gains obtained with MAG-RAG compared to pure LLM. (D) Frequency of scoring gains obtained with pure MA compared to pure LLM.