Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

NOMTO: Neural Operator-based symbolic Model approximaTion and discOvery

Sergei Garmaev, Siddhartha Mishra, Olga Fink

TL;DR

NOMTO addresses the limitation of traditional symbolic regression, which is restricted to a narrow set of algebraic functions, by leveraging neural operators to auto-approximate a broader library of symbolic operations, including derivatives and special functions. The method builds a computational graph where neural operator surrogates implement library operations, and learns sparse edge weights to produce compact, interpretable symbolic expressions; it can also rediscover governing PDEs directly from data. Empirical results show NOMTO yields competitive reconstruction of benchmark expressions, demonstrates the discovery of derivatives and special functions, and successfully recovers the two-dimensional heat and Burgers' equations, with performance depending on the surrogate (FNO vs. CNO) used. This approach broadens the applicability of symbolic regression to complex physical systems, enabling data-driven discovery of nonlinear models and PDEs with potentially wide impact in physics, engineering, and beyond.

Abstract

While many physical and engineering processes are most effectively described by non-linear symbolic models, existing non-linear symbolic regression (SR) methods are restricted to a limited set of continuous algebraic functions, thereby limiting their applicability to discover higher order non-linear differential relations. In this work, we introduce the Neural Operator-based symbolic Model approximaTion and discOvery (NOMTO) method, a novel approach to symbolic model discovery that leverages Neural Operators to encompass a broad range of symbolic operations. We demonstrate that NOMTO can successfully identify symbolic expressions containing elementary functions with singularities, special functions, and derivatives. Additionally, our experiments demonstrate that NOMTO can accurately rediscover second-order non-linear partial differential equations. By broadening the set of symbolic operations available for discovery, NOMTO significantly advances the capabilities of existing SR methods. It provides a powerful and flexible tool for model discovery, capable of capturing complex relations in a variety of physical systems.

NOMTO: Neural Operator-based symbolic Model approximaTion and discOvery

TL;DR

NOMTO addresses the limitation of traditional symbolic regression, which is restricted to a narrow set of algebraic functions, by leveraging neural operators to auto-approximate a broader library of symbolic operations, including derivatives and special functions. The method builds a computational graph where neural operator surrogates implement library operations, and learns sparse edge weights to produce compact, interpretable symbolic expressions; it can also rediscover governing PDEs directly from data. Empirical results show NOMTO yields competitive reconstruction of benchmark expressions, demonstrates the discovery of derivatives and special functions, and successfully recovers the two-dimensional heat and Burgers' equations, with performance depending on the surrogate (FNO vs. CNO) used. This approach broadens the applicability of symbolic regression to complex physical systems, enabling data-driven discovery of nonlinear models and PDEs with potentially wide impact in physics, engineering, and beyond.

Abstract

While many physical and engineering processes are most effectively described by non-linear symbolic models, existing non-linear symbolic regression (SR) methods are restricted to a limited set of continuous algebraic functions, thereby limiting their applicability to discover higher order non-linear differential relations. In this work, we introduce the Neural Operator-based symbolic Model approximaTion and discOvery (NOMTO) method, a novel approach to symbolic model discovery that leverages Neural Operators to encompass a broad range of symbolic operations. We demonstrate that NOMTO can successfully identify symbolic expressions containing elementary functions with singularities, special functions, and derivatives. Additionally, our experiments demonstrate that NOMTO can accurately rediscover second-order non-linear partial differential equations. By broadening the set of symbolic operations available for discovery, NOMTO significantly advances the capabilities of existing SR methods. It provides a powerful and flexible tool for model discovery, capable of capturing complex relations in a variety of physical systems.
Paper Structure (19 sections, 13 equations, 4 figures, 5 tables)

This paper contains 19 sections, 13 equations, 4 figures, 5 tables.

Table of Contents

  1. Main
  2. Results
  3. Symbolic Regression Task
  4. Expressions with Derivatives and Special Functions
  5. Governing equations rediscovery
  6. Two-dimensional Heat Equation
  7. Two-dimensional Burgers' Equation
  8. Discussion
  9. Methodology
  10. Expression Tree and Library
  11. Neural Operators as Surrogate Library Operations
  12. Composing NOMTO Computational Graph
  13. Optimization and Expression Extraction
  14. Data availability
  15. Acknowledgements
  16. ...and 4 more sections

Figures (4)

  • Figure 1: The general implementation steps and architecture of the Neural Operator-based symbolic Model approximaTion and discOvery (NOMTO) algorithm.
  • Figure 2: Heat equation rediscovery results: The top row presents the outcomes from NOMTO-FNO; while the bottom row shows results from NOMTO-CNO. Panel (a) displays the ground truth $\frac{\partial u}{\partial t}$ obtained through numerical simulation. Panel (b) illustrates the $\frac{\partial u}{\partial t}$ predictions generated by the NOMTO algorithm. Panel (c) depicts the MSE distribution across the simulation domain, highlighting regions of higher and lower prediction accuracy.
  • Figure 3: Construction of the Neural Operator Block.
  • Figure 4: Prediction MSE distributions within the simulation domain: NOMTO-FNO for $\frac{\partial u}{\partial t}$ (a) and $\frac{\partial v}{\partial t}$ (b); NOMTO-CNO $\frac{\partial u}{\partial t}$ (c) and $\frac{\partial v}{\partial t}$ (d).