Thinking Outside the Template with Modular GP-GOMEA

TL;DR

This work addresses the trade-off between accuracy and interpretability in Symbolic Regression by extending GP-GOMEA with Modular GP-GOMEA, a multi-tree template framework that explicitly evolves and reuses subexpressions as functional components. By constraining each subexpression to a small template while allowing cross-tree references, the method increases representational flexibility without excessive bloating, improving both ground-truth recovery on synthetic data and predictive performance on real-world datasets. The experiments show that modular templates generally outperform single-template GP-GOMEA, with larger gains when more multi-trees and deeper templates are used within a reasonable time budget; qualitative analysis also suggests interpretability benefits from subexpression reuse. The work points to future improvements via parsimony pressure and multi-objectivization to further encourage useful subexpression reuse and maintain interpretability in larger, more complex expressions.

Abstract

The goal in Symbolic Regression (SR) is to discover expressions that accurately map input to output data. Because often the intent is to understand these expressions, there is a trade-off between accuracy and the interpretability of expressions. GP-GOMEA excels at producing small SR expressions (increasing the potential for interpretability) with high accuracy, but requires a fixed tree template, which limits the types of expressions that can be evolved. This paper presents a modular representation for GP-GOMEA that allows multiple trees to be evolved simultaneously that can be used as (functional) subexpressions. While each tree individually is constrained to a (small) fixed tree template, the final expression, if expanded, can exhibit a much larger structure. Furthermore, the use of subexpressions decomposes the original regression problem and opens the possibility for enhanced interpretability through the piece-wise understanding of small subexpressions. We compare the performance of GP-GOMEA with and without modular templates on a variety of datasets. We find that our proposed approach generally outperforms single-template GP-GOMEA and can moreover uncover ground-truth expressions underlying synthetic datasets with modular subexpressions at a faster rate than GP-GOMEA without modular subexpressions.

Figures (10)

• Figure 1: Example of an FOS and a GOM swap. The red rectangle indicates an FOS subset that undergoes GOM. Node indices (in pre-order) are indicated with a subscript.
• Figure 2: Example of an individual with three trees $t_0$, $t_1$, and $t_2$. $t_2$ is the output of the tree. $t_1$ is used as a function by $t_2$. The argument nodes in a tree $i$ are indicated by $a_n$ and point to the arguments in the tree $j$ ($j>i$) that calls the tree $i$ (i.e, as a function), e.g. $a_0$ points to $x_1$ for the leftmost call to tree (function) $t_1$ in tree $t_2$.
• Figure 3: Comparison of average $R^2$ between different configurations of GP-GOMEA (4 trees of depth 4, 1 tree of depth 7) and depth-constrained GP. Positions of the markers of the GP configurations are shifted sideways for clarity.
• Figure 4: The population size with the highest average $R^2$ overall runs over time for the 4x4 modular configuration of GP-GOMEA. $R^2$ is interpolated linearly to obtain specific time points. The overall average is calculated as the average population size over the highest averages. At the 1-hour mark, the average population size is 5324.
• Figure 5: Comparison between the ability of different configurations of GP-GOMEA (4 tree templates of depth 4, 1 tree template of depth 7) to recover expressions.