ISR: Invertible Symbolic Regression

TL;DR

This work addresses inverse problems by learning interpretable, invertible symbolic mappings that yield tractable posterior inference. By integrating Equation Learner with Invertible Neural Network blocks, ISR provides a bijective, differentiable framework that produces symbolic forward and inverse relationships, along with a conditional variant (cISR) for observation-conditioned mappings. The approach is demonstrated as a symbolic normalizing flow for density estimation and applied to inverse kinematics and geoacoustic inversion, producing competitive posteriors and explicit symbolic expressions. The results highlight the potential of interpretable, symbolic invertible maps for robust inverse modeling and uncertainty quantification in scientific domains.

Abstract

We introduce an Invertible Symbolic Regression (ISR) method. It is a machine learning technique that generates analytical relationships between inputs and outputs of a given dataset via invertible maps (or architectures). The proposed ISR method naturally combines the principles of Invertible Neural Networks (INNs) and Equation Learner (EQL), a neural network-based symbolic architecture for function learning. In particular, we transform the affine coupling blocks of INNs into a symbolic framework, resulting in an end-to-end differentiable symbolic invertible architecture that allows for efficient gradient-based learning. The proposed ISR framework also relies on sparsity promoting regularization, allowing the discovery of concise and interpretable invertible expressions. We show that ISR can serve as a (symbolic) normalizing flow for density estimation tasks. Furthermore, we highlight its practical applicability in solving inverse problems, including a benchmark inverse kinematics problem, and notably, a geoacoustic inversion problem in oceanography aimed at inferring posterior distributions of underlying seabed parameters from acoustic signals.

Figures (9)

• Figure 1: EQL network architecture for symbolic regression. For visual simplicity, we only show 2 hidden layers and 5 activation functions per layer (identity or "id", square, sine, exponential, and multiplication).
• Figure 2: (Left) The proposed ISR framework learns a bijective symbolic transformation that maps the (unknown) variables $\mathbf{x}$ to the (observed) quantities $\mathbf{y}$ while transforming the lost information into latent variables $\mathbf{z}$. (Right) The conditional ISR (cISR) framework learns a bijective symbolic map that transforms $\mathbf{x}$ directly to a latent representation $\mathbf{z}$ given the observation $\mathbf{y}$. As we will show, both the forward and inverse mappings are efficiently computable and possess a tractable Jacobian, allowing explicit computation of posterior probabilities.
• Figure 3: The proposed ISR method integrates EQL within the affine coupling blocks of the INN invertible architecture. This results in a bijective symbolic transformation that is both easily invertible and has a tractable Jacobian. Indeed, the forward and inverse directions both possess identical computational cost. Here, $\odot$ and $\oslash$ denote element-wise multiplication and divison, respectively.
• Figure 4: Samples from four different target densities (first row), and their estimated distributions using INN (second row) and the proposed ISR method (third row).
• Figure 5: Results for the inverse kinematics benchmark problem. The faint colored lines indicate sampled arm configurations $\mathbf{x}$ taken from each model's predicted posterior $\hat{p}(\mathbf{x}\,|\,\mathbf{y}^*)$, conditioned on the target end point $\mathbf{y}^*$, which is indicated by a gray cross. The contour lines around the target end point enclose the regions containing 97% of the sampled arms’ end points. We emphasize the arm with the highest estimated likelihood as a bold line.

Theorems & Definitions (5)

• Remark 3.1
• Remark 3.2
• Remark 3.3
• Remark 3.4
• Remark 3.5