Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SymPlex: A Structure-Aware Transformer for Symbolic PDE Solving

Yesom Park, Annie C. Lu, Shao-Ching Huang, Qiyang Hu, Y. Sungtaek Ju, Stanley Osher

TL;DR

SymPlex formulates symbolic PDE solving as a tree-structured reinforcement learning problem and introduces SymFormer, a structure-aware Transformer that uses tree-relative self-attention and grammar-constrained autoregressive decoding to generate valid symbolic expressions. By optimizing a PDE-based reward and employing a curriculum learning strategy, it can recover exact analytical solutions, including non-smooth and parametric ones, while providing explicit parametric dependencies. Theoretical guarantees establish conditions under which exact symbolic recovery is possible, and experiments demonstrate superior symbolic recovery and interpretability compared with numerical solvers and neural baselines. This framework offers a principled, human-readable alternative to discretized or purely neural PDE solvers, with potential impact on inverse problems, control, and analytical understanding of PDEs, while leaving open avenues for scaling to high-dimensional problems and vocabulary design.

Abstract

We propose SymPlex, a reinforcement learning framework for discovering analytical symbolic solutions to partial differential equations (PDEs) without access to ground-truth expressions. SymPlex formulates symbolic PDE solving as tree-structured decision-making and optimizes candidate solutions using only the PDE and its boundary conditions. At its core is SymFormer, a structure-aware Transformer that models hierarchical symbolic dependencies via tree-relative self-attention and enforces syntactic validity through grammar-constrained autoregressive decoding, overcoming the limited expressivity of sequence-based generators. Unlike numerical and neural approaches that approximate solutions in discretized or implicit function spaces, SymPlex operates directly in symbolic expression space, enabling interpretable and human-readable solutions that naturally represent non-smooth behavior and explicit parametric dependence. Empirical results demonstrate exact recovery of non-smooth and parametric PDE solutions using deep learning-based symbolic methods.

SymPlex: A Structure-Aware Transformer for Symbolic PDE Solving

TL;DR

SymPlex formulates symbolic PDE solving as a tree-structured reinforcement learning problem and introduces SymFormer, a structure-aware Transformer that uses tree-relative self-attention and grammar-constrained autoregressive decoding to generate valid symbolic expressions. By optimizing a PDE-based reward and employing a curriculum learning strategy, it can recover exact analytical solutions, including non-smooth and parametric ones, while providing explicit parametric dependencies. Theoretical guarantees establish conditions under which exact symbolic recovery is possible, and experiments demonstrate superior symbolic recovery and interpretability compared with numerical solvers and neural baselines. This framework offers a principled, human-readable alternative to discretized or purely neural PDE solvers, with potential impact on inverse problems, control, and analytical understanding of PDEs, while leaving open avenues for scaling to high-dimensional problems and vocabulary design.

Abstract

We propose SymPlex, a reinforcement learning framework for discovering analytical symbolic solutions to partial differential equations (PDEs) without access to ground-truth expressions. SymPlex formulates symbolic PDE solving as tree-structured decision-making and optimizes candidate solutions using only the PDE and its boundary conditions. At its core is SymFormer, a structure-aware Transformer that models hierarchical symbolic dependencies via tree-relative self-attention and enforces syntactic validity through grammar-constrained autoregressive decoding, overcoming the limited expressivity of sequence-based generators. Unlike numerical and neural approaches that approximate solutions in discretized or implicit function spaces, SymPlex operates directly in symbolic expression space, enabling interpretable and human-readable solutions that naturally represent non-smooth behavior and explicit parametric dependence. Empirical results demonstrate exact recovery of non-smooth and parametric PDE solutions using deep learning-based symbolic methods.
Paper Structure (57 sections, 5 theorems, 48 equations, 3 figures, 6 tables, 1 algorithm)

This paper contains 57 sections, 5 theorems, 48 equations, 3 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Symbolic PDE and expression discovery.
  4. Tree-structured and grammar-aware Transformers.
  5. Preliminary
  6. Parametric PDEs
  7. Symbolic Expression Representation
  8. SymFormer: Structure-Aware Attention for Symbolic Expressions
  9. Tree-Relative Self-Attention for Hierarchical Reasoning
  10. Traversal-Aware Positional Encoding
  11. Grammar-Constrained Autoregressive Generation
  12. Structure-Conditioned Expressive Power of SymFormer
  13. Reinforcement Learning for Symbolic PDE Solutions
  14. PDE-Aware Reward
  15. Policy Loss with Entropy and Imitation
  16. ...and 42 more sections

Key Result

Theorem 4.1

Any grammar-compatible next-token policy that depends only on the structure of a partial abstract syntax tree of bounded depth can be represented by SymFormer. Equivalently, SymFormer can realize any decision rule defined over symbolic tree states, independent of the particular linear prefix represe

Figures (3)

  • Figure 1: Comparison of discontinuous solutions of a Hamilton-Jacobi PDE using grid-based, neural network, and our symbolic method. The symbolic approach achieves exact solutions with minimal storage memory, whereas other methods exhibit larger errors and higher storage requirements.
  • Figure 2: Left: Standard attention ignores tree structure. Right: SymFormer explicitly models hierarchical relations.
  • Figure 3: Reward evolution of SymPlex with and without curriculum learning for two parametric PDEs in Table \ref{['tab:summary_pdes']}. The single-stage setting learns the full problem at once without curriculum learning, while curriculum learning trains a single SymFormer sequentially through Stages 1, 2, and 3. The results show stable optimization across stages under the proposed curriculum.

Theorems & Definitions (8)

  • Theorem 4.1: Informal
  • Theorem 5.1: Symbolic Recovery by SymPlex
  • Theorem 4.1: Structure-Conditioned Policy Universality of SymFormer
  • proof
  • Theorem 4.2: Conditional Exact Symbolic Recovery by SymPlex
  • proof
  • Theorem 4.3: Near-Optimal Probabilistic Recovery
  • proof