CodePDE: An Inference Framework for LLM-driven PDE Solver Generation

TL;DR

CodePDE reframes PDE solving as executable solver-code generation guided by LLMs, enabling automated reasoning, debugging, self-refinement, and test-time scaling without domain-specific training. The framework demonstrates near-human to superhuman performance across five PDE families, often surpassing human expert solvers, and provides detailed analyses of numerical schemes, libraries, and failure modes. A key contribution is the systematic study of LLM-generated solvers, including bug-free rates, refinement gains, and scaling laws, supported by a public leaderboard and modular Python implementation. While promising, the approach reveals limitations on Reaction-Diffusion problems and highlights interpretability as a major advantage for debugging and improvement. The work points to a future where LLM-driven solver generation complements traditional methods and neural solvers, with potential improvements from domain-focused data, external numerical verification, and hybrid models.

Abstract

Partial differential equations (PDEs) are fundamental to modeling physical systems, yet solving them remains a complex challenge. Traditional numerical solvers rely on expert knowledge to implement and are computationally expensive, while neural-network-based solvers require large training datasets and often lack interpretability. In this work, we frame PDE solving as a code generation task and introduce CodePDE, the first inference framework for generating PDE solvers using large language models (LLMs). Leveraging advanced inference-time algorithms and scaling strategies, CodePDE unlocks critical capacities of LLM for PDE solving: reasoning, debugging, selfrefinement, and test-time scaling -- all without task-specific tuning. CodePDE achieves superhuman performance across a range of representative PDE problems. We also present a systematic empirical analysis of LLM generated solvers, analyzing their accuracy, efficiency, and numerical scheme choices. Our findings highlight the promise and the current limitations of LLMs in PDE solving, offering a new perspective on solver design and opportunities for future model development. Our code is available at https://github.com/LithiumDA/CodePDE.

Figures (16)

• Figure 1: CodePDE achieves near-human performance via LLM-driven PDE solver generation. The plot visualizes normalized $-\log(\text{nRMSE})$ across PDE families (higher is better).
• Figure 2: An overview of CodePDE framework. Critical LLM capabilities are unlocked in the steps: code generation leverages chain-of-thought reasoning, code repair enables autonomous debugging, evaluation implements test-time scaling through best-of-n sampling, and solver refinement optimizes performance through feedback-driven improvement.
• Figure 3: Bug-free rates before and after introducing automated debugging. Left: Averaged across all five PDE datasets for each model. Right: Averaged across all LLMs for each PDE family.
• Figure 4: Smoothed test-time scaling curves for representative models on each PDE family. We increase $n$ in the best-of-$n$ sampling to study test-time scaling performance.
• Figure 5: Convergence rates across LLMs and PDEs. Higher orders imply faster convergence.