Curiosity-driven RL for symbolic equation solving

TL;DR

This paper investigates whether reinforcement learning can aid symbolic mathematics by solving nonlinear one-variable equations such as $a x + b = 0$, $a x^2 + b x + c = 0$, and equations involving radicals, exponentials, and trig functions, using curiosity-driven PPO with a graph-based action space. By representing equations as expression trees and enabling macro actions (Expand, Collect, multiply by $-1$) and a rich set of operations, the authors show that curiosity-guided exploration helps the agent learn to solve these forms and generalize to novel instances. Compared with prior work that used text-based states or linear forms, this approach provides dense rewards and a structured action space that improves performance on harder tasks, though it remains limited to 'closed' equations solvable within the original term set. The study suggests that combining exploration with structured, high-level actions offers a path toward more robust symbolic reasoning and could underpin future work on differential equations and other advanced reasoning tasks.

Abstract

We explore if RL can be useful for symbolic mathematics. Previous work showed contrastive learning can solve linear equations in one variable. We show model-free PPO \cite{schulman2017proximal} augmented with curiosity-based exploration and graph-based actions can solve nonlinear equations such as those involving radicals, exponentials, and trig functions. Our work suggests curiosity-based exploration may be useful for general symbolic reasoning tasks.

Figures (2)

• Figure 1: Expression trees for $ax+b$ and $ax^2+bx+c$.
• Figure 2: Top: small dataset; Bottom: large dataset. Curves show mean success rate over 3 trials; shading spans min–max range.