Beyond Error-Based Optimization: Experience-Driven Symbolic Regression with Goal-Conditioned Reinforcement Learning
Jianwen Sun, Xinrui Li, Fuqing Li, Xiaoxuan Shen
TL;DR
This work tackles the limitations of error-based symbolic regression by reframing SR as a goal-conditioned reinforcement learning problem. It introduces EGRL-SR, which leverages hindsight experience replay and a structure-aware exploration strategy to learn reusable $x$-$y$ mapping patterns rather than optimizing solely for local error minima. The All-Point Satisfaction Reward (APSR) and Structure-Guided Heuristic Exploration (SGHE) components, together with a Double Dueling DQN, guide exploration toward structurally valid, robust expressions and enable better recovery on challenging targets, especially under noise. Empirical results on Nguyen, Livermore, and Keijzer benchmarks show higher exact recovery rates and robustness compared with baselines, with ablations confirming the critical roles of the action-value network, reward design, and exploration strategy.
Abstract
Symbolic Regression aims to automatically identify compact and interpretable mathematical expressions that model the functional relationship between input and output variables. Most existing search-based symbolic regression methods typically rely on the fitting error to inform the search process. However, in the vast expression space, numerous candidate expressions may exhibit similar error values while differing substantially in structure, leading to ambiguous search directions and hindering convergence to the underlying true function. To address this challenge, we propose a novel framework named EGRL-SR (Experience-driven Goal-conditioned Reinforcement Learning for Symbolic Regression). In contrast to traditional error-driven approaches, EGRL-SR introduces a new perspective: leveraging precise historical trajectories and optimizing the action-value network to proactively guide the search process, thereby achieving a more robust expression search. Specifically, we formulate symbolic regression as a goal-conditioned reinforcement learning problem and incorporate hindsight experience replay, allowing the action-value network to generalize common mapping patterns from diverse input-output pairs. Moreover, we design an all-point satisfaction binary reward function that encourages the action-value network to focus on structural patterns rather than low-error expressions, and concurrently propose a structure-guided heuristic exploration strategy to enhance search diversity and space coverage. Experiments on public benchmarks show that EGRL-SR consistently outperforms state-of-the-art methods in recovery rate and robustness, and can recover more complex expressions under the same search budget. Ablation results validate that the action-value network effectively guides the search, with both the reward function and the exploration strategy playing critical roles.