Cluster-Based Generalized Additive Models Informed by Random Fourier Features

TL;DR

This work addresses the tension between predictive accuracy and interpretability in regression by proposing a mixture of generalized additive models (GAMs) guided by random Fourier feature (RFF) representations. An RFF embedding is learned and compressed with PCA, then soft-clustered via a Gaussian mixture model; cluster-specific GAMs are fitted to capture nonlinear effects in each region, with the final predictor a weighted sum over clusters. Empirical results on California Housing, Airfoil Self-Noise, and Bike Sharing show that the proposed approach surpasses classical interpretable baselines and closely tracks or competes with RFF-based methods, while offering transparent, per-cluster nonlinear effects. The method provides a principled way to combine representation learning with transparent statistical modeling, yielding interpretable insights such as spatial or spectral structure driving cluster formation.

Abstract

Explainable machine learning aims to strike a balance between prediction accuracy and model transparency, particularly in settings where black-box predictive models, such as deep neural networks or kernel-based methods, achieve strong empirical performance but remain difficult to interpret. This work introduces a mixture of generalized additive models (GAMs) in which random Fourier feature (RFF) representations are leveraged to uncover locally adaptive structure in the data. In the proposed method, an RFF-based embedding is first learned and then compressed via principal component analysis. The resulting low-dimensional representations are used to perform soft clustering of the data through a Gaussian mixture model. These cluster assignments are then applied to construct a mixture-of-GAMs framework, where each local GAM captures nonlinear effects through interpretable univariate smooth functions. Numerical experiments on real-world regression benchmarks, including the California Housing, NASA Airfoil Self-Noise, and Bike Sharing datasets, demonstrate improved predictive performance relative to classical interpretable models. Overall, this construction provides a principled approach for integrating representation learning with transparent statistical modeling.

Figures (13)

• Figure 1: The overall pipeline of the RFF-informed mixture-of-GAMs. Stage 1 learns RFF model coefficients and builds a random Fourier feature space representation. Stage 2 reduces to a lower-dimensional representation via PCA and fits a GMM to obtain posterior responsibilities. Stage 3 trains local GAMs for each cluster of the training dataset. Stage 4 forms the final prediction.
• Figure 2: Diagram with graphical representations of the workflow of the mixture-of-GAMs method informed with random Fourier features.
• Figure 3: Root mean square error of the trained random Fourier feature model $\bar{m}(\bm{x})$, with increasing number of resampling iterations on the random frequency samples.
• Figure 4: Test root mean square error on California housing dataset evaluated over a grid of hyperparameter configurations $(L, d)$, where $L$ is the number of mixture components and $d$ is the number of retained principal components after PCA on the intermediate feature representations. Panel (a) uses the full set of RFF features to guide the Gaussian mixture model-based clustering, while panel (b) relies solely on spatial RFF features derived from geographic coordinates.
• Figure 5: Partial dependence plots for selected features of the California Housing dataset.