Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PINNsAgent: Automated PDE Surrogation with Large Language Models

Qingpo Wuwu, Chonghan Gao, Tianyu Chen, Yihang Huang, Yuekai Zhang, Jianing Wang, Jianxin Li, Haoyi Zhou, Shanghang Zhang

TL;DR

PINNsAgent introduces an LLM-driven surrogate framework for automated PINN design to solve PDEs, addressing the expertise gap in hyperparameter tuning and architecture selection. It combines Physics-Guided Knowledge Replay (PGKR) for knowledge transfer from solved PDEs with a Memory Tree Reasoning Strategy (MTRS) that guides exploration of the PINN space. The approach operates in two modes (Config Generation and Code Generation) and leverages a Code Bank and iterative revisions to refine configurations over multiple iterations. Empirical results on the PINNacle 14 PDEs show strong performance gains over random and Bayesian baselines, with ablations confirming the essential roles of PGKR and MTRS in achieving robust, high-accuracy surrogates.

Abstract

Solving partial differential equations (PDEs) using neural methods has been a long-standing scientific and engineering research pursuit. Physics-Informed Neural Networks (PINNs) have emerged as a promising alternative to traditional numerical methods for solving PDEs. However, the gap between domain-specific knowledge and deep learning expertise often limits the practical application of PINNs. Previous works typically involve manually conducting extensive PINNs experiments and summarizing heuristic rules for hyperparameter tuning. In this work, we introduce PINNsAgent, a novel surrogation framework that leverages large language models (LLMs) and utilizes PINNs as a foundation to bridge the gap between domain-specific knowledge and deep learning. Specifically, PINNsAgent integrates (1) Physics-Guided Knowledge Replay (PGKR), which encodes the essential characteristics of PDEs and their associated best-performing PINNs configurations into a structured format, enabling efficient knowledge transfer from solved PDEs to similar problems and (2) Memory Tree Reasoning, a strategy that effectively explores the search space for optimal PINNs architectures. By leveraging LLMs and exploration strategies, PINNsAgent enhances the automation and efficiency of PINNs-based solutions. We evaluate PINNsAgent on 14 benchmark PDEs, demonstrating its effectiveness in automating the surrogation process and significantly improving the accuracy of PINNs-based solutions.

PINNsAgent: Automated PDE Surrogation with Large Language Models

TL;DR

PINNsAgent introduces an LLM-driven surrogate framework for automated PINN design to solve PDEs, addressing the expertise gap in hyperparameter tuning and architecture selection. It combines Physics-Guided Knowledge Replay (PGKR) for knowledge transfer from solved PDEs with a Memory Tree Reasoning Strategy (MTRS) that guides exploration of the PINN space. The approach operates in two modes (Config Generation and Code Generation) and leverages a Code Bank and iterative revisions to refine configurations over multiple iterations. Empirical results on the PINNacle 14 PDEs show strong performance gains over random and Bayesian baselines, with ablations confirming the essential roles of PGKR and MTRS in achieving robust, high-accuracy surrogates.

Abstract

Solving partial differential equations (PDEs) using neural methods has been a long-standing scientific and engineering research pursuit. Physics-Informed Neural Networks (PINNs) have emerged as a promising alternative to traditional numerical methods for solving PDEs. However, the gap between domain-specific knowledge and deep learning expertise often limits the practical application of PINNs. Previous works typically involve manually conducting extensive PINNs experiments and summarizing heuristic rules for hyperparameter tuning. In this work, we introduce PINNsAgent, a novel surrogation framework that leverages large language models (LLMs) and utilizes PINNs as a foundation to bridge the gap between domain-specific knowledge and deep learning. Specifically, PINNsAgent integrates (1) Physics-Guided Knowledge Replay (PGKR), which encodes the essential characteristics of PDEs and their associated best-performing PINNs configurations into a structured format, enabling efficient knowledge transfer from solved PDEs to similar problems and (2) Memory Tree Reasoning, a strategy that effectively explores the search space for optimal PINNs architectures. By leveraging LLMs and exploration strategies, PINNsAgent enhances the automation and efficiency of PINNs-based solutions. We evaluate PINNsAgent on 14 benchmark PDEs, demonstrating its effectiveness in automating the surrogation process and significantly improving the accuracy of PINNs-based solutions.
Paper Structure (29 sections, 7 equations, 2 figures, 3 tables)

This paper contains 29 sections, 7 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Learned PDE Solvers
  4. LLM-based Autonomous SciML Agents
  5. LLMs enabled AutoML
  6. Methods
  7. Preliminary
  8. Problem Formulation: Solving PDEs with PINNs
  9. Hyperparameter Optimization via LLM
  10. PINNsAgent
  11. Step 1: Database Retrieval
  12. Step 2: Experiment Plan Generation
  13. Step 3: Code Execution
  14. Step 4: Revision
  15. Database Exploitation: Physics-Guided Knowledge Replay (PGKR)
  16. ...and 14 more sections

Figures (2)

  • Figure 1: The workflow of the PINNsAgent's Framework. The PINNsAgent operates in two modes: Code Generation and Config Generation. It leverages LLM agents to generate and refine executable code and YAML configuration files for optimizing hyperparameters in PINNs. The planner and programmer collaborate to devise experimental plans and generate training code, utilizing a central Code Bank and top-K cases from the Database.
  • Figure 2: Memory Tree Reasoning Strategy. The root node represents the corresponding PDE, with subsequent levels corresponding to different hyperparameters. The planner selects unexplored branches and generates configurations, which are executed to obtain MSE scores. This process iterates to refine the tree and find the global optimal architecture (Arch 8 with the lowest MSE).