A reproducible comparative study of categorical kernels for Gaussian process regression, with new clustering-based nested kernels

TL;DR

This work addresses the challenge of selecting categorical kernels for Gaussian process regression with mixed inputs by delivering a large, reproducible benchmark of 23 kernels across 42 datasets. It introduces two main kernel construction paradigms (encoding-based and covariance-parameterization) and proposes a clustering-based, target-encoding method to infer group structure when unknown. Key findings show nested kernels with well-chosen within/between structures consistently outperform alternatives when groups exist, and that automatic group inference via target encoding (MSD) or LVGP initialization remains competitive even without prior group information. The results emphasize that hypersphere-based kernels struggle under default optimization settings and that robust, scalable performance is achieved by nested kernels with appropriate initialization, providing actionable guidance for practitioners.

Abstract

Designing categorical kernels is a major challenge for Gaussian process regression with continuous and categorical inputs. Despite previous studies, it is difficult to identify a preferred method, either because the evaluation metrics, the optimization procedure, or the datasets change depending on the study. In particular, reproducible code is rarely available. The aim of this paper is to provide a reproducible comparative study of all existing categorical kernels on many of the test cases investigated so far. We also propose new evaluation metrics inspired by the optimization community, which provide quantitative rankings of the methods across several tasks. From our results on datasets which exhibit a group structure on the levels of categorical inputs, it appears that nested kernels methods clearly outperform all competitors. When the group structure is unknown or when there is no prior knowledge of such a structure, we propose a new clustering-based strategy using target encodings of categorical variables. We show that on a large panel of datasets, which do not necessarily have a known group structure, this estimation strategy still outperforms other approaches while maintaining low computational cost.

Figures (16)

• Figure 1: Number of parameters of several categorical kernels. $C$ : number of levels, $q$: latent dimension (for encoding-based or low-rank approaches), $\gamma$: number of groups (for nested kernels), $n_l$: number of levels per group (for nested kernels). Remark that $C=\sum_{l=1}^\gamma n_l$ so that we always have $C \leq \sum_{l=1}^\gamma n_l^2$ and $\sum_{l=1}^\gamma n_l^2 \leq C^2 \leq \gamma \sum_{l=1}^\gamma n_l^2$.
• Figure 2: Three datasets with known group structure. Boxplots of the RRSME over the 50 experiments for all methods, "long" optimization setting.
• Figure 3: Performance profiles using the RRMSE for datasets with a group structure, ”long” optimization setting. The score after the name of the method $i$ is $\mathrm{AUC}(p_i)$, methods are sorted in AUC decreasing order.
• Figure 4: Target encoding illustration. The z-axis represents 3 levels, and for each of them the conditional distribution of the output is represented on the y-axis. Intuitively, levels 1 and 2 should be grouped together since their target encoding distributions are close.
• Figure 5: Target distributions of the 12 levels of the beam bending problem. From left to right: 3/6/9/12/15 samples by level. Each color represents a different group.