Introns and Templates Matter: Rethinking Linkage in GP-GOMEA

TL;DR

This paper addresses how introns and a fixed template in GP-GOMEA distort linkage learning based on mutual information in symbolic regression. It proposes two measures: masking inactive introns during MI estimation (MI_masked) and a template-driven node proximity similarity (Node) derived from the predefined template. Across five symbolic regression benchmarks, both measures substantially improve performance, with the template-aligned Node measure delivering the best results. The findings show that explicitly exploiting known problem structure and inactive-variable information can yield faster convergence and more interpretable, compact expressions.

Abstract

GP-GOMEA is among the state-of-the-art for symbolic regression, especially when it comes to finding small and potentially interpretable solutions. A key mechanism employed in any GOMEA variant is the exploitation of linkage, the dependencies between variables, to ensure efficient evolution. In GP-GOMEA, mutual information between node positions in GP trees has so far been used to learn linkage. For this, a fixed expression template is used. This however leads to introns for expressions smaller than the full template. As introns have no impact on fitness, their occurrences are not directly linked to selection. Consequently, introns can adversely affect the extent to which mutual information captures dependencies between tree nodes. To overcome this, we propose two new measures for linkage learning, one that explicitly considers introns in mutual information estimates, and one that revisits linkage learning in GP-GOMEA from a grey-box perspective, yielding a measure that needs not to be learned from the population but is derived directly from the template. Across five standard symbolic regression problems, GP-GOMEA achieves substantial improvements using both measures. We also find that the newly learned linkage structure closely reflects the template linkage structure, and that explicitly using the template structure yields the best performance overall.

Figures (17)

• Figure 1: A fixed-length string of decision variables is mapped to a fixed tree template, which defines the corresponding semantic expression. Shaded variables are introns, and do not affect the semantic meaning of the expression.
• Figure 2: Top left: population of 4 solutions. Bottom left: tree template. Matrices in blue show entropy (above the black line) and MI (below the black line). The corresponding linkage tree for both the normal and proposed masked MI are shown as well. The masked version treats all introns (shaded) as the same separate "masked" symbol.
• Figure 3: An example template with highlighted subfunctions and the corresponding node proximity and subfunction based similarity measures.
• Figure 4: The interquartile mean (IQM) training $R^2$ and bootstrapped 95% confidence intervals (as per agarwalDeepReinforcementLearning) over evaluations and approximate runtime, across all runs performed. For the runtime, the last recorded value before each point in time was used, and dashed horizontal lines show the final values reached for the existing measures. The vertical line indicates the computational budget (not to scale w.r.t runtime) where the final values reached are marked with crosses.
• Figure 5: Aggregate $R^2$ scores (higher is better) on the problems for each combination of template height and LS considered. The interquartile mean corresponds to the mean after discarding the bottom and top 25% of runs for each problem, and the mean of medians corresponds to the mean of the median performances on each problem. The colored bar corresponds to the 95% confidence interval estimated using a percentile bootstrap with stratified sampling as per agarwalDeepReinforcementLearning with the expected value in black.