Improving the efficiency of GP-GOMEA for higher-arity operators

TL;DR

The paper addresses the scalability of GP-GOMEA when using higher-arity operators by introducing semantic subtree inheritance and greedy child selection, along with an expanded set of operators. By enforcing type-like constraints and exploiting introns through targeted variation, the approach improves search efficiency on both continuous and discontinuous symbolic regression tasks, particularly as operator-set size and arity grow. Empirical results on Feynman-equation benchmarks show consistent gains in performance and faster convergence, with SSI providing broad benefits and ternary operators enhancing expressiveness. The work advances interpretable modeling in GP for complex, piecewise relationships, with implications for explainable AI and more efficient symbolic regression in real-world domains.

Abstract

Deploying machine learning models into sensitive domains in our society requires these models to be explainable. Genetic Programming (GP) can offer a way to evolve inherently interpretable expressions. GP-GOMEA is a form of GP that has been found particularly effective at evolving expressions that are accurate yet of limited size and, thus, promote interpretability. Despite this strength, a limitation of GP-GOMEA is template-based. This negatively affects its scalability regarding the arity of operators that can be used, since with increasing operator arity, an increasingly large part of the template tends to go unused. In this paper, we therefore propose two enhancements to GP-GOMEA: (i) semantic subtree inheritance, which performs additional variation steps that consider the semantic context of a subtree, and (ii) greedy child selection, which explicitly considers parts of the template that in standard GP-GOMEA remain unused. We compare different versions of GP-GOMEA regarding search enhancements on a set of continuous and discontinuous regression problems, with varying tree depths and operator sets. Experimental results show that both proposed search enhancements have a generally positive impact on the performance of GP-GOMEA, especially when the set of operators to choose from is large and contains higher-arity operators.

Figures (6)

• Figure 1: Two trees, each describing the Gaussian probability density function. \ref{['fig:example_small']} makes use of a high-level operator.
• Figure 2: Semantic Subtree Inheritance: For each non-leaf, non-intron node, the node and its subtree are inherited based on its operator from a random donor individual from the population that includes the same operator somewhere in its tree. The ticks and crosses indicate that a change was accepted or rejected, respectively. The faint grey nodes indicate a subtree filled with syntactic introns.
• Figure 3: Greedy Child Selection: Nodes are visited in a depth-first traversal through the tree to select the child(ren) and their order with the best fit for the visited non-leaf, non-intron node. An optional step is to also visit a subtree that was previously unvisited as its root was an intron, which has now been selected to be used for its parent node. The figure shows this process for a tree with a branching factor of three. Thick lines indicate selected children, whereas dotted lines refer to intron children. The nodes show the following arities: 13, 4, and 12 are of arity 2, whereas 8 is of arity 1.
• Figure 4: Statistical significant differences between the mean rank of GP-GOMEA configurations. Two configurations that are connected by a bar are not significantly different, whereas configurations that are further apart than the critical distance (CD) are statistically significantly different.
• Figure 5: The impact of varying the operators on the R2 metrics of different GP-GOMEA configurations on continuous and discontinuous problems of different depths. The performances are measured after 5,000,000 function evaluations. A B indicates a binary tree, whereas a T indicates a ternary tree. Points indicate the mean of the median R2 performances per dataset, whereas the bars indicate the 95% confidence interval. To ease comparison, the red vertical line shows the mean performance of the original GP-GOMEA configuration in the respective setting.