Structured Radial Basis Function Network: Modelling Diversity for Multiple Hypotheses Prediction

TL;DR

The paper tackles multi-modal regression by proposing the Structured Radial Basis Function Network (s-RBFN), an ensemble that couples multiple hypotheses with a structured dataset derived from centroidal Voronoi tessellations. Base predictors are trained to form tessellations, then their predictions generate a structured dataset $D(\varepsilon)$, from which an RBFN with Gaussian bases is trained in closed-form via least squares, enabling efficient multi-hypotheses predictions. A diversity control mechanism, via the parameter $\varepsilon$, shapes the tessellations to balance diversity and generalization. Empirical evaluation on air quality and energy appliance datasets demonstrates that s-RBFN achieves superior generalization and computational efficiency compared to single-hypothesis baselines and standard MHP approaches, with clear evidence of optimal diversity settings and regularization effects guiding robust performance. The work suggests that carefully managing the number of hypotheses and the diversity parameter yields strong generalization, and points to future extensions with deeper architectures and multi-modal data.

Abstract

Multi-modal problems can be effectively addressed using multiple hypothesis frameworks, but integrating these frameworks into learning models poses significant challenges. This paper introduces a Structured Radial Basis Function Network (s-RBFN) as an ensemble of multiple hypothesis predictors for regression. During the training of the predictors, first the centroidal Voronoi tessellations are formed based on their losses and the true labels, representing geometrically the set of multiple hypotheses. Then, the trained predictors are used to compute a structured dataset with their predictions, including centers and scales for the basis functions. A radial basis function network, with each basis function focused on a particular hypothesis, is subsequently trained using this structured dataset for multiple hypotheses prediction. The s-RBFN is designed to train efficiently while controlling diversity in ensemble learning parametrically. The least-squares approach for training the structured ensemble model provides a closed-form solution for multiple hypotheses and structured predictions. During the formation of the structured dataset, a parameter is employed to avoid mode collapse by controlling tessellation shapes. This parameter provides a mechanism to balance diversity and generalization performance for the s-RBFN. The empirical validation on two multivariate prediction datasets-air quality and energy appliance predictions-demonstrates the superior generalization performance and computational efficiency of the structured ensemble model compared to other models and their single-hypothesis counterparts.

Figures (7)

• Figure 1: Model architecture with structured data obtained from neural networks' predictions and the ground-truth labels $\boldsymbol{y}$ forming centroidal Voronoi tessellations based on the neural networks' losses (Left box). The s-RBFN uses these predictions to estimate the ground-truth labels $\hat{\boldsymbol{y}}$, with $L$ representing the s-RBFN norm loss (Right Box).
• Figure 2: Air quality test set: Mean RMSE and $90\%$ confidence interval from 10-fold cross-validation for different configurations for hyper-parameters $M$ and $\varepsilon$.
• Figure 3: Energy appliances test set: Mean RMSE and $90\%$ confidence interval from 10-fold cross-validation for different configurations for hyper-parameters $M$ and $\varepsilon$.
• Figure 4: Mean RMSE
• Figure 5: Standard deviation RMSE