[RETRACTED]Evolving Form and Function: Dual-Objective Optimization in Neural Symbolic Regression Networks

TL;DR

The paper tackles the challenge of symbolic regression generalization by separating form and function metrics and proposing a dual-objective SR method, SRNE, that evolves neural networks to minimize both symbolic loss $CE(eq,\hat{eq})$ and numeric loss $MSE(Y,\hat{Y})$. By pretraining with gradient descent and then applying evolutionary weight optimization, SRNE creates data-to-equation networks that outperform state-of-the-art GNSR methods on unseen equations, achieving near-zero NMSE and near-zero CE on test sets. The key contribution is demonstrating that joint optimization of symbolic and numeric objectives via neuroevolution yields more accurate and generalizable equations, suggesting a productive synergy between evolutionary algorithms and gradient-based training for SR. The work has practical significance for automated, interpretable modeling across diverse datasets, offering a path toward robust equation discovery without manual form specification or per-dataset retraining.

Abstract

[RETRACTED]Data increasingly abounds, but distilling their underlying relationships down to something interpretable remains challenging. One approach is genetic programming, which `symbolically regresses' a data set down into an equation. However, symbolic regression (SR) faces the issue of requiring training from scratch for each new dataset. To generalize across all datasets, deep learning techniques have been applied to SR. These networks, however, are only able to be trained using a symbolic objective: NN-generated and target equations are symbolically compared. But this does not consider the predictive power of these equations, which could be measured by a behavioral objective that compares the generated equation's predictions to actual data. Here we introduce a method that combines gradient descent and evolutionary computation to yield neural networks that minimize the symbolic and behavioral errors of the equations they generate from data. As a result, these evolved networks are shown to generate more symbolically and behaviorally accurate equations than those generated by networks trained by state-of-the-art gradient based neural symbolic regression methods. We hope this method suggests that evolutionary algorithms, combined with gradient descent, can improve SR results by yielding equations with more accurate form and function.

Figures (6)

• Figure 1: Emergence of valid equations. Blue is the percentage of the 20 trials that have started producing valid equations, enabling dual-objective optimization. Orange is the percentage of proportion of one trial that are producing invalid equations over the generations. Only 50 generations are shown for readability.
• Figure 2: Average symbolic loss over 20 trials. Horizontal axis shows generations and the vertical axis is cross-entropy (CE) loss. A 95% confidence interval is shown. Vertical axis is logarithmic for readability.
• Figure 3: Average numeric loss over 20 trials. Horizontal axis shows generations and the vertical axis is mean squared error (MSE) loss. A 95% confidence interval is shown. Vertical axis is logarithmic for readability on the outer plot, but is linear on the inset plot.
• Figure 4: (a): Pareto-fronts for the population of evolved networks at the final generation over 20 trials, evaluated on the training equations. Black is the meta-Pareto drawn from the combination of all trials' populations. The horizontal axis is logarithmic for readability. (b): Performance of other GNSR methods from literature compared to SRNE (ours) networks on unseen test equations. Compared methods are averaged over the same 100 test equations. Zoomed window (lower right) shows the meta Pareto-front of the SNRE networks, assembled from 15 final evolved networks from 20 trials that are evaluated after evolution on the testing data-equation pairs.
• Figure 5: Examples of various evolved SRNE networks and cases where said networks were unable to perfectly predict the target equation. The equations show the target (blue) and predicted (orange) equations. The plots show the target $Y$ data (blue) and the predicted $\hat{Y}$ (orange).