ROIDS: Robust Outlier-Aware Informed Down-Sampling

TL;DR

ROIDS addresses the brittleness of Informed Down-Sampling (IDS) in symbolic regression when data contain outliers. By excluding high-mean-error cases from the sampling step, ROIDS preserves IDS’s ability to sample informative edge cases while reducing overfitting to outliers. Across synthetic benchmarks with outliers and real-world regression problems, ROIDS achieves superior or comparable performance to IDS and frequently outperforms it, with an average ranking of 1.7. The approach requires minimal additional cost and offers a robust, practical down-sampling option for genetic programming in symbolic regression.

Abstract

Informed down-sampling (IDS) is known to improve performance in symbolic regression when combined with various selection strategies, especially tournament selection. However, recent work found that IDS's gains are not consistent across all problems. Our analysis reveals that IDS performance is worse for problems containing outliers. IDS systematically favors including outliers in subsets which pushes GP towards finding solutions that overfit to outliers. To address this, we introduce ROIDS (Robust Outlier-Aware Informed Down-Sampling), which excludes potential outliers from the sampling process of IDS. With ROIDS it is possible to keep the advantages of IDS without overfitting to outliers and to compete on a wide range of benchmark problems. This is also reflected in our experiments in which ROIDS shows the desired behavior on all studied benchmark problems. ROIDS consistently outperforms IDS on synthetic problems with added outliers as well as on a wide range of complex real-world problems, surpassing IDS on over 80% of the real-world benchmark problems. Moreover, compared to all studied baseline approaches, ROIDS achieves the best average rank across all tested benchmark problems. This robust behavior makes ROIDS a reliable down-sampling method for selection in symbolic regression, especially when outliers may be included in the data set.

Figures (16)

• Figure 1: Color-coded frequencies of including a training case in the subsets when using IDS. Results are for different variants of the nguyen-6 problem. The top row contains two outlier-free variants of the problem, whereas the bottom row includes 5% outliers.
• Figure 2: Color-coded frequencies of including a training case in the subsets when using ROIDS. Results are for different variants of the nguyen-6 problem. The top row contains two outlier-free variants of the problem, whereas the bottom row includes 5% outliers.
• Figure 3: Performance of different down-sampling methods for different variants of the nguyen-6 problem. Note that the y-axes of the plots are scaled differently. For better readability, outliers are not plotted.
• Figure 4: Performance of different down-sampling methods for Friedman problems with and without outliers. Note that the y-axes of the plots are scaled differently. For better readability, outliers are not plotted.
• Figure 5: Performance of different down-sampling methods for real-world problems. For better readability, outliers are not plotted.