Nonparametric contaminated Gaussian mixture of regressions

TL;DR

The paper addresses robustness in finite mixtures of regressions by introducing non-parametric contaminated Gaussian mixtures (NPCGMRs) and semi-parametric contaminated Gaussian mixtures (SPCGMRs). It develops backfitting-based estimation with local-likelihood for non-parametric terms and EM/ECM variants to estimate CG parameters, enabling simultaneous clustering and outlier detection. Through extensive simulations and a real-data application, SPCGMRs consistently outperform Gaussian-based counterparts in the presence of mild outliers, while NPCGMRs offer strong robustness and flexibility for non-parametric components. The work provides practical model-based clustering, outlier detection, and data-driven component selection, with guidance for extensions to higher-dimensional covariates and more complex non-parametric structures.

Abstract

Semi- and non-parametric mixture of regressions are a very useful flexible class of mixture of regressions in which some or all of the parameters are non-parametric functions of the covariates. These models are, however, based on the Gaussian assumption of the component error distributions. Thus, their estimation is sensitive to outliers and heavy-tailed error distributions. In this paper, we propose semi- and non-parametric contaminated Gaussian mixture of regressions to robustly estimate the parametric and/or non-parametric terms of the models in the presence of mild outliers. The virtue of using a contaminated Gaussian error distribution is that we can simultaneously perform model-based clustering of observations and model-based outlier detection. We propose two algorithms, an expectation-maximization (EM)-type algorithm and an expectation-conditional-maximization (ECM)-type algorithm, to perform maximum likelihood and local-likelihood kernel estimation of the parametric and non-parametric of the proposed models, respectively. The robustness of the proposed models is examined using an extensive simulation study. The practical utility of the proposed models is demonstrated using real data.

Figures (6)

• Figure 1: Scatter plots (left-panel) and Histograms (right-panel) of random samples of size $n=250$ generated from each of the Scenarios of Experiment 1. Overlaid on each scatter plot, are the component regression functions.
• Figure 2: Scatter plot of the HPI data
• Figure 3: The fitted component regression functions (black solid lines) for the original HPI data: (a) SPGMRs model (b) SPCGMRs model. The MAP approach was used to classify points into the two components. Points with a symbol $\bullet$ are in component 1 and points with a symbol $\blacktriangle$ are in component 2
• Figure 4: The fitted component regression functions (black solid lines) for the HPI data with $5$ outliers $(0.6,2.5)$: (a) SPGMRs model (b) SPCGMRs model. The MAP approach was used to classify points into the two components. Points with a symbol $\bullet$ are in component 1 and points with a symbol $\blacktriangle$ are in component 2
• Figure 5: The fitted component regression functions (black solid lines) for the HPI data with $5$ outliers $(0.6,3.5)$: (a) SPGMRs model (b) SPCGMRs model. The MAP approach was used to classify points into the two components. Points with a symbol $\bullet$ are in component 1 and points with a symbol $\blacktriangle$ are in component 2