Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Call for Action: towards the next generation of symbolic regression benchmark

Guilherme S. Imai Aldeia, Hengzhe Zhang, Geoffrey Bomarito, Miles Cranmer, Alcides Fonseca, Bogdan Burlacu, William G. La Cava, Fabrício Olivetti de França

TL;DR

This paper addresses the need for a mature, reproducible benchmark for symbolic regression by presenting SRBench 2.0, an expanded and standardized evaluation framework. It increases the number of evaluated algorithms to 25, extends datasets to 24 across two tracks, and employs rigorous, resource-constrained benchmarking with grid-search hyperparameter tuning and energy accounting. The results demonstrate that no single SR method dominates across all problems and highlight trade-offs between accuracy, model size, and energy use, motivating a living benchmark approach with deprecation rules and adaptive practices. The work's significance lies in providing a scalable, transparent, and community-driven platform to accelerate progress in symbolic regression and to guide the design of more robust, energy-efficient algorithms.

Abstract

Symbolic Regression (SR) is a powerful technique for discovering interpretable mathematical expressions. However, benchmarking SR methods remains challenging due to the diversity of algorithms, datasets, and evaluation criteria. In this work, we present an updated version of SRBench. Our benchmark expands the previous one by nearly doubling the number of evaluated methods, refining evaluation metrics, and using improved visualizations of the results to understand the performances. Additionally, we analyze trade-offs between model complexity, accuracy, and energy consumption. Our results show that no single algorithm dominates across all datasets. We propose a call for action from SR community in maintaining and evolving SRBench as a living benchmark that reflects the state-of-the-art in symbolic regression, by standardizing hyperparameter tuning, execution constraints, and computational resource allocation. We also propose deprecation criteria to maintain the benchmark's relevance and discuss best practices for improving SR algorithms, such as adaptive hyperparameter tuning and energy-efficient implementations.

Call for Action: towards the next generation of symbolic regression benchmark

TL;DR

This paper addresses the need for a mature, reproducible benchmark for symbolic regression by presenting SRBench 2.0, an expanded and standardized evaluation framework. It increases the number of evaluated algorithms to 25, extends datasets to 24 across two tracks, and employs rigorous, resource-constrained benchmarking with grid-search hyperparameter tuning and energy accounting. The results demonstrate that no single SR method dominates across all problems and highlight trade-offs between accuracy, model size, and energy use, motivating a living benchmark approach with deprecation rules and adaptive practices. The work's significance lies in providing a scalable, transparent, and community-driven platform to accelerate progress in symbolic regression and to guide the design of more robust, energy-efficient algorithms.

Abstract

Symbolic Regression (SR) is a powerful technique for discovering interpretable mathematical expressions. However, benchmarking SR methods remains challenging due to the diversity of algorithms, datasets, and evaluation criteria. In this work, we present an updated version of SRBench. Our benchmark expands the previous one by nearly doubling the number of evaluated methods, refining evaluation metrics, and using improved visualizations of the results to understand the performances. Additionally, we analyze trade-offs between model complexity, accuracy, and energy consumption. Our results show that no single algorithm dominates across all datasets. We propose a call for action from SR community in maintaining and evolving SRBench as a living benchmark that reflects the state-of-the-art in symbolic regression, by standardizing hyperparameter tuning, execution constraints, and computational resource allocation. We also propose deprecation criteria to maintain the benchmark's relevance and discuss best practices for improving SR algorithms, such as adaptive hyperparameter tuning and energy-efficient implementations.
Paper Structure (16 sections, 4 figures, 3 tables)

This paper contains 16 sections, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Methods
  4. Dataset selection
  5. Benchmarked methods
  6. Performance analysis
  7. Benchmark infrastructure
  8. Data Availability
  9. Results
  10. Discussion
  11. Energy consumption and execution time
  12. Performance analysis of the black-box track
  13. Towards a parameter-less experience
  14. Solving empirical and theoretical problems
  15. Current difficulties and limitations
  16. ...and 1 more sections

Figures (4)

  • Figure 1: Median energy consumption (kWh) and training runtime for each algorithm.
  • Figure 2: Performance plots for the black-box track, where the lines represent the probability of obtaining a given empirically observed $R^2$ value when running the experiments multiple time (i.e., max aggregation).
  • Figure 3: Cluster map of the Area Under the Curve (AUC) of Expected Performances across the $30$ independent runs is segregated by algorithm and dataset. Higher values indicate better performance, while larger cells represent worse model size.
  • Figure 4: Pareto plots for the phenomenological & first-principles track, with model sizes on $y$ axis, and $R^2$ on $x$ axis. The star marker denotes the ground-truth expression performance, also denoted in the box inside each subplot. Only the first front is plotted for clarity.