Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

UniSymNet: A Unified Symbolic Network Guided by Transformer

Xinxin Li, Juan Zhang, Da Li, Xingyu Liu, Jin Xu, Junping Yin

TL;DR

This paper tackles the challenge of interpretable and scalable Symbolic Regression by introducing UniSymNet, a Unified Symbolic Network that rewrites nonlinear binary operators using nested unary operators via $\ln$ and $\exp$, enabling multivariate interactions with reduced depth and node count. A Transformer-based framework with a novel label-encoding guides structure discovery, while objective-specific optimization (including Gumbel-Softmax sampling, risk-seeking policy gradients, and BFGS) tunes both structure and parameters. The method demonstrates strong fitting accuracy, high symbolic-solution rates, and favorable expression complexity on low-dimensional Standard Benchmarks and high-dimensional SRBench, with ablations showing the value of structure encoding, pruning, and optimization choices. The work advances interpretability and extrapolation in SR, offering a scalable approach that blends learned structure with principled optimization to discover compact, accurate mathematical expressions across diverse domains.

Abstract

Symbolic Regression (SR) is a powerful technique for automatically discovering mathematical expressions from input data. Mainstream SR algorithms search for the optimal symbolic tree in a vast function space, but the increasing complexity of the tree structure limits their performance. Inspired by neural networks, symbolic networks have emerged as a promising new paradigm. However, most existing symbolic networks still face certain challenges: binary nonlinear operators $\{\times, ÷\}$ cannot be naturally extended to multivariate operators, and training with fixed architecture often leads to higher complexity and overfitting. In this work, we propose a Unified Symbolic Network that unifies nonlinear binary operators into nested unary operators and define the conditions under which UniSymNet can reduce complexity. Moreover, we pre-train a Transformer model with a novel label encoding method to guide structural selection, and adopt objective-specific optimization strategies to learn the parameters of the symbolic network. UniSymNet shows high fitting accuracy, excellent symbolic solution rate, and relatively low expression complexity, achieving competitive performance on low-dimensional Standard Benchmarks and high-dimensional SRBench.

UniSymNet: A Unified Symbolic Network Guided by Transformer

TL;DR

This paper tackles the challenge of interpretable and scalable Symbolic Regression by introducing UniSymNet, a Unified Symbolic Network that rewrites nonlinear binary operators using nested unary operators via and , enabling multivariate interactions with reduced depth and node count. A Transformer-based framework with a novel label-encoding guides structure discovery, while objective-specific optimization (including Gumbel-Softmax sampling, risk-seeking policy gradients, and BFGS) tunes both structure and parameters. The method demonstrates strong fitting accuracy, high symbolic-solution rates, and favorable expression complexity on low-dimensional Standard Benchmarks and high-dimensional SRBench, with ablations showing the value of structure encoding, pruning, and optimization choices. The work advances interpretability and extrapolation in SR, offering a scalable approach that blends learned structure with principled optimization to discover compact, accurate mathematical expressions across diverse domains.

Abstract

Symbolic Regression (SR) is a powerful technique for automatically discovering mathematical expressions from input data. Mainstream SR algorithms search for the optimal symbolic tree in a vast function space, but the increasing complexity of the tree structure limits their performance. Inspired by neural networks, symbolic networks have emerged as a promising new paradigm. However, most existing symbolic networks still face certain challenges: binary nonlinear operators cannot be naturally extended to multivariate operators, and training with fixed architecture often leads to higher complexity and overfitting. In this work, we propose a Unified Symbolic Network that unifies nonlinear binary operators into nested unary operators and define the conditions under which UniSymNet can reduce complexity. Moreover, we pre-train a Transformer model with a novel label encoding method to guide structural selection, and adopt objective-specific optimization strategies to learn the parameters of the symbolic network. UniSymNet shows high fitting accuracy, excellent symbolic solution rate, and relatively low expression complexity, achieving competitive performance on low-dimensional Standard Benchmarks and high-dimensional SRBench.
Paper Structure (46 sections, 3 theorems, 30 equations, 17 figures, 8 tables, 2 algorithms)

This paper contains 46 sections, 3 theorems, 30 equations, 17 figures, 8 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Genetic Programming for Symbolic Regression
  4. Neural Networks for Symbolic Regression
  5. Transformer Model for Symbolic Regression
  6. Methods
  7. Structural Space Formalization
  8. Structure-to-Sequence Encoding
  9. UniSymNet Structure Inference
  10. Data Generation
  11. Training
  12. Symbolic Function Optimization
  13. Differentiable Network Optimization
  14. Experiments
  15. Metrics
  16. ...and 31 more sections

Key Result

Theorem 1

Let $P(\mathbf{x}) = \sum_{k=1}^{K} c_k \prod_{i=1}^{d} x_i^{m_{k i}}$, where $c_k \in \mathbb{R}$, $m_{k i} \in \mathbb{R} \setminus \{0\}$ and $d,k\in \mathbb{N}$. Suppose that $\exists (k_0, i_0)$, $s.t. \ m_{k_0 i_0} \neq 1$. Then, for any function $P(\mathbf{x})$, there exists a UniSymNet archi

Figures (17)

  • Figure 1: The overall architecture of EQL$^\div$.
  • Figure 2: The overall architecture of our method.
  • Figure 3: The overall architecture of UniSymNet.
  • Figure 4: The process of simplifying UniSymNet's structure.
  • Figure 5: Performance comparison of two encoding methods as the number of hidden layers in the equation varies.
  • ...and 12 more figures

Theorems & Definitions (10)

  • Theorem 1
  • Theorem 2
  • definition 1
  • definition 2
  • definition 3
  • definition 4
  • Lemma 1
  • proof
  • proof
  • proof