Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enabling Large Language Models to Perform Power System Simulations with Previously Unseen Tools: A Case of Daline

Mengshuo Jia, Zeyu Cui, Gabriela Hug

TL;DR

A modular framework that integrates expertise from both the power system and LLM domains enhances LLMs' ability to perform power system simulations on previously unseen tools and highlights the potential of LLMs as research assistants in power systems.

Abstract

The integration of experiment technologies with large language models (LLMs) is transforming scientific research, offering AI capabilities beyond specialized problem-solving to becoming research assistants for human scientists. In power systems, simulations are essential for research. However, LLMs face significant challenges in power system simulations due to limited pre-existing knowledge and the complexity of power grids. To address this issue, this work proposes a modular framework that integrates expertise from both the power system and LLM domains. This framework enhances LLMs' ability to perform power system simulations on previously unseen tools. Validated using 34 simulation tasks in Daline, a (optimal) power flow simulation and linearization toolbox not yet exposed to LLMs, the proposed framework improved GPT-4o's simulation coding accuracy from 0% to 96.07%, also outperforming the ChatGPT-4o web interface's 33.8% accuracy (with the entire knowledge base uploaded). These results highlight the potential of LLMs as research assistants in power systems.

Enabling Large Language Models to Perform Power System Simulations with Previously Unseen Tools: A Case of Daline

TL;DR

A modular framework that integrates expertise from both the power system and LLM domains enhances LLMs' ability to perform power system simulations on previously unseen tools and highlights the potential of LLMs as research assistants in power systems.

Abstract

The integration of experiment technologies with large language models (LLMs) is transforming scientific research, offering AI capabilities beyond specialized problem-solving to becoming research assistants for human scientists. In power systems, simulations are essential for research. However, LLMs face significant challenges in power system simulations due to limited pre-existing knowledge and the complexity of power grids. To address this issue, this work proposes a modular framework that integrates expertise from both the power system and LLM domains. This framework enhances LLMs' ability to perform power system simulations on previously unseen tools. Validated using 34 simulation tasks in Daline, a (optimal) power flow simulation and linearization toolbox not yet exposed to LLMs, the proposed framework improved GPT-4o's simulation coding accuracy from 0% to 96.07%, also outperforming the ChatGPT-4o web interface's 33.8% accuracy (with the entire knowledge base uploaded). These results highlight the potential of LLMs as research assistants in power systems.

Paper Structure

This paper contains 10 sections, 4 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Proposed Modular Framework
  3. Prompt Engineering
  4. Enhanced RAG
  5. Toolbox Refinement
  6. Feedback Loop
  7. Case Study
  8. Configuration
  9. Accuracy Analysis and Discussion
  10. Conclusion

Figures (4)

  • Figure 1: Proposed framework with techniques indexed from 1 to 10. $N$ is the number of feedback iterations and $N_{max}$ is the maximum number of iterations.
  • Figure 2: Score achieved by every scheme in each attempt when processing an example request (i.e., normal task 20, as given in Table \ref{['table:example']}).
  • Figure 3: Overall accuracy of evaluated schemes across both complex and normal tasks (the feedback loop is enabled for all schemes).
  • Figure 4: Individual accuracy of evaluated schemes given the complex or the normal tasks, respectively (the feedback loop is enabled for all schemes).